Tag: Model Serving

2 articles

• LLM LoRA Adapter Cache in Production: Stabilizing Multi-Tenant Fine-Tuning Services with Heat Tiering, Cold-Start Budgets, and Eviction Policies

  Production strategies for LoRA adapter cache management in multi-tenant fine-tuning services: heat tiering, cold-start budgets, eviction policies, and a governance framework for stable model serving.

  07/09/2026 LLM Serving / LoRA / Adapter Cache / Multi Tenant / Model Serving / GPU / Fine Tuning / RAG / vLLM

• LLM Quantization in Practice: FP8, INT4, AWQ, and Production Inference Trade-offs

  A systematic guide to LLM inference quantization: FP8, INT4, AWQ, GPTQ, KV Cache quantization, benefits, risks, selection methods, and a go-live checklist for production engineers.

  06/29/2026 LLM Inference / Quantization / vLLM / TensorRT-LLM / Model Serving