Tag: awq

2 articles

• LLM Weight Quantization in Production: Preventing Quality Regression with Calibration Sets, FP8/W4A16 Dual-Track Benchmarks, and Quality Gates

  LLM weight quantization is more than just reduced VRAM. This guide covers calibration set selection, FP8/W4A16 dual-track benchmarking, hardware matching, quality gates, and canary rollouts using vLLM, LLM Compressor, and TensorRT-LLM.

  08/31/2026 llm-serving / quantization / fp8 / awq / model-deployment

• LLM Weight Quantization in Production: Ensuring Quality and Throughput with Calibration Fingerprints, Kernel Compatibility Matrices, and Dual-Track Replay

  A deep dive into the complete pipeline for deploying LLM weight quantization from experimentation to production, covering AWQ/GPTQ algorithm comparisons, calibration set fingerprinting, a four-layer compatibility model, kernel compatibility matrices, dual-track quality and throughput replay, and canary release/rollback strategies.

  07/12/2026 LLM Inference / Model Quantization / AWQ / GPTQ / GPU Serving / Weight Quantization