Articles

Searchable, citable, durable content on LLMs and AI agents. 147 articles, page 15 / 15.

• Structured Outputs Are Not JSON Prompts: The Engineering Boundaries of Constrained Decoding

  Explains how structured outputs improve reliability via JSON Schema, constrained decoding, and frameworks like vLLM and Outlines, plus latency, schema design, and fallback considerations for production.

  06/29/2026 LLM / Structured Outputs / Constrained Decoding / JSON Schema

• KV Cache Reuse in Practice: Prefix Caching, Offloading, and Long-Context Cost Management

  A systematic guide to KV Cache reuse for long-context LLM serving: Prefix Caching, RadixAttention, multi-tier offloading, hit rate, latency, cost, and production checklists.

  06/29/2026 LLM Inference / KV Cache / Prefix Caching / vLLM / SGLang

• Speculative Decoding in Production: Why Low QPS Speeds Up but High Load Slows Down

  A production-oriented analysis of speculative decoding's benefit boundaries, covering draft models, verification, acceptance rate, request load, and batching, with a pre-deployment checklist.

  06/29/2026 LLM Inference / Speculative Decoding / vLLM / TensorRT-LLM / Performance Optimization

• Prefill-Decode Disaggregation in Practice: Taming Latency for Long-Context LLM Serving

  Long contexts and agentic requests shift LLM inference bottlenecks from single-card throughput to tail-latency governance. This post dives into Prefill-Decode disaggregation, KV cache transfer, resource planning, and a pre-launch checklist to move LLM serving from 'working' to stable and scalable.

  06/29/2026 LLM Serving / Prefill Decode / KV Cache / Inference Optimization / Long Context / Tail Latency / RAG / Agent

• Prefill-Decode Disaggregation in Practice: Taming Latency for Long-Context LLM Serving

  Long contexts and agentic requests shift the LLM inference bottleneck from single-GPU throughput to tail-latency management. This article dives into the principles, trade-offs, KV cache transfer, resource planning, and a go-live checklist for Prefill-Decode disaggregation.

  06/29/2026 LLM Serving / Prefill-Decode / KV Cache / Inference Optimization / Long Context / Latency Management

• LLM Quantization in Practice: FP8, INT4, AWQ, and Production Inference Trade-offs

  A systematic guide to LLM inference quantization: FP8, INT4, AWQ, GPTQ, KV Cache quantization, benefits, risks, selection methods, and a go-live checklist for production engineers.

  06/29/2026 LLM Inference / Quantization / vLLM / TensorRT-LLM / Model Serving

• PagedAttention in Action: Why KV Cache Scheduling Matters More Than Compute for LLM Serving

  A deep dive into KV Cache memory fragmentation, PagedAttention, continuous batching, and prefix caching — and how they boost throughput and reduce latency in LLM inference.

  06/29/2026 LLM Inference / PagedAttention / KV Cache / vLLM / LLM Engineering