Tag: fp8

2 articles

• LLM Weight Quantization in Production: Preventing Quality Regression with Calibration Sets, FP8/W4A16 Dual-Track Benchmarks, and Quality Gates

  LLM weight quantization is more than just reduced VRAM. This guide covers calibration set selection, FP8/W4A16 dual-track benchmarking, hardware matching, quality gates, and canary rollouts using vLLM, LLM Compressor, and TensorRT-LLM.

  08/31/2026 llm-serving / quantization / fp8 / awq / model-deployment

• LLM FP8 Fine-Tuning in Production: Amax Scaling, Layer Whitelists, and BF16 Fallback for Stable Convergence

  FP8 reduces compute and memory for LLM fine-tuning, but poor scaling, outliers, and distributed sync issues can cause silent accuracy degradation. This guide provides a complete engineering solution with Amax monitoring, layer whitelists, BF16 fallback, dual-track evaluation, and release gates.

  07/14/2026 LLM Training / FP8 / Transformer Engine / Fine-tuning / MLOps / Mixed Precision Training / Distributed Training / Amax Scaling