Siyu Ding

Full-Stack FP4 Narrows Pretraining Loss Gap

Module-specific NVFP4 quantization covers projections, optimizers, and attention, reaching a 1.47% loss gap on 3B/64B-token pretraining.