Articles

Searchable, citable, durable content on LLMs and AI agents. 128 articles, page 13 / 13.

• Prompt Caching in Practice: Cost and Latency Governance for Long-Context Agents

  A deep dive into Prompt Caching principles and engineering practices—prefix stability, cache boundaries, and agent scenarios—with reproducible optimizations and a production checklist to cut input costs and TTFT without changing model semantics.

  06/29/2026 LLM / Prompt Caching / AI Agent / Cost Optimization / Context Engineering

• Structured Outputs Are Not JSON Prompts: The Engineering Boundaries of Constrained Decoding

  Explains how structured outputs improve reliability via JSON Schema, constrained decoding, and frameworks like vLLM and Outlines, plus latency, schema design, and fallback considerations for production.

  06/29/2026 LLM / Structured Outputs / Constrained Decoding / JSON Schema

• KV Cache Reuse in Practice: Prefix Caching, Offloading, and Long-Context Cost Management

  A systematic guide to KV Cache reuse for long-context LLM serving: Prefix Caching, RadixAttention, multi-tier offloading, hit rate, latency, cost, and production checklists.

  06/29/2026 LLM Inference / KV Cache / Prefix Caching / vLLM / SGLang

• Speculative Decoding in Production: Why Low QPS Speeds Up but High Load Slows Down

  A production-oriented analysis of speculative decoding's benefit boundaries, covering draft models, verification, acceptance rate, request load, and batching, with a pre-deployment checklist.

  06/29/2026 LLM Inference / Speculative Decoding / vLLM / TensorRT-LLM / Performance Optimization

• Prefill-Decode Disaggregation in Practice: Taming Latency for Long-Context LLM Serving

  Long contexts and agentic requests shift LLM inference bottlenecks from single-card throughput to tail-latency governance. This post dives into Prefill-Decode disaggregation, KV cache transfer, resource planning, and a pre-launch checklist to move LLM serving from 'working' to stable and scalable.

  06/29/2026 LLM Serving / Prefill Decode / KV Cache / Inference Optimization / Long Context / Tail Latency / RAG / Agent

• Prefill-Decode Disaggregation in Practice: Taming Latency for Long-Context LLM Serving

  Long contexts and agentic requests shift the LLM inference bottleneck from single-GPU throughput to tail-latency management. This article dives into the principles, trade-offs, KV cache transfer, resource planning, and a go-live checklist for Prefill-Decode disaggregation.

  06/29/2026 LLM Serving / Prefill-Decode / KV Cache / Inference Optimization / Long Context / Latency Management

• LLM Quantization in Practice: FP8, INT4, AWQ, and Production Inference Trade-offs

  A systematic guide to LLM inference quantization: FP8, INT4, AWQ, GPTQ, KV Cache quantization, benefits, risks, selection methods, and a go-live checklist for production engineers.

  06/29/2026 LLM Inference / Quantization / vLLM / TensorRT-LLM / Model Serving

• PagedAttention in Action: Why KV Cache Scheduling Matters More Than Compute for LLM Serving

  A deep dive into KV Cache memory fragmentation, PagedAttention, continuous batching, and prefix caching — and how they boost throughput and reduce latency in LLM inference.

  06/29/2026 LLM Inference / PagedAttention / KV Cache / vLLM / LLM Engineering