Tag: Model Compression

2 articles

• Production 2:4 Structured Sparsity for LLMs: Turning Pruning into Real Speedups with Prune Gates, Tactic Audits, and Dense Fallbacks

  A production guide to 2:4 structured sparsity in LLMs—using prune gates, tactic audits, and dense fallbacks to turn pruning into real GPU speedups.

  09/22/2026 LLM Serving / Structured Sparsity / GPU Inference / Model Compression / TensorRT

• LLM Knowledge Distillation in Production: Controlling Student Model Degradation with Teacher Logprob Contracts, On-Policy Replay, and Top-K Tail Buckets

  A practical engineering guide to productionizing LLM knowledge distillation, covering Teacher Logprob contracts, temperature and divergence selection, On-policy replay, Top-K tail probability buckets, Tokenizer alignment, and student capability gating to prevent silent degradation and deployment failures.

  07/28/2026 LLM / Knowledge Distillation / Model Compression / On-policy Distillation / MLOps