Articles

Searchable, citable, durable content on LLMs and AI agents. 147 articles, page 13 / 15.

• PagedAttention in Production: Boosting LLM Serving Throughput with Paged KV Cache

  A deep dive into how PagedAttention mitigates GPU memory fragmentation and over-reservation in LLM inference via paged KV cache management, helping engineering teams increase throughput, reduce latency variance, and providing a production deployment checklist and monitoring strategy.

  06/30/2026 llm-serving / pagedattention / kv-cache / inference

• LLM Observability in Production: Tracing Agent Call Chains with OpenTelemetry GenAI Semantic Conventions

  A systematic guide to building observability for LLM and Agent applications using OpenTelemetry GenAI semantic conventions. Covers trace chains, tokens, tool calls, privacy governance, and pre-launch checks to help teams pinpoint latency, cost, and quality issues.

  06/30/2026 llm / observability / opentelemetry / ai-agent / rag / genai / agent-tracing

• MCP Tool Poisoning in Practice: Building Secure Tool Call Boundaries for LLM Agents

  A systematic breakdown of MCP tool poisoning and indirect prompt injection risks, covering a five-layer defense architecture from tool registration, least privilege, parameter auditing, content isolation to runtime interception, with a go-live checklist for teams integrating external APIs, databases, and enterprise systems.

  06/30/2026 llm-agent / mcp / ai-security / prompt-injection / tool-poisoning

• Prefix Caching in Production: A Complete Guide to Reusing KV Cache for RAG and Agent Services

  A systematic guide to Prefix Caching for reusing KV Cache of shared prompts and contexts, reducing TTFT, inference costs, and redundant prefill in RAG, Agent, and long-document QA scenarios. Includes engineering checklists for vLLM, OpenAI API, and SGLang.

  06/30/2026 LLM Serving / KV Cache / RAG / Agent Infrastructure / Prefix Caching / Inference Optimization

• Structured Outputs in Production: Stabilizing JSON Schema Generation with Constrained Decoding

  A systematic guide to Structured Outputs and Constrained Decoding: core principles, schema design, server integration, performance, error handling, and common pitfalls for reliable LLM JSON generation.

  06/30/2026 LLM Engineering / Structured Outputs / Constrained Decoding / JSON Schema

• Agent Memory Engineering: Building a Governable Memory Layer for LLM Agents

  From short-term sessions to long-term memory, systematically deconstruct the write, manage, read, forget, and evaluate mechanisms for LLM Agents. Build a production-ready memory system that avoids the pitfalls of an uncontrollable vector store.

  06/30/2026 LLM Agent / Agent Memory / RAG / LangGraph / LlamaIndex / LangMem / MemGPT / Vector Database / Memory Governance

• Contextual Retrieval in Production: Making RAG Chunks Retrievable with Context

  A practical guide to Contextual Retrieval: indexing with context, hybrid search, reranking, and deployment checks to reduce missed recall and improve answer traceability.

  06/30/2026 RAG / Contextual Retrieval / Hybrid Search / Reranking / Vector Search

• Speculative Decoding in Production: Reducing LLM Per-User Latency with EAGLE and Draft Models

  A practical guide to Speculative Decoding for LLM inference: draft generation, parallel verification, acceptance length, and production metrics. Compares Draft Model, Medusa, and EAGLE with rollout and rollback checklists.

  06/30/2026 LLM Inference / Speculative Decoding / vLLM / EAGLE

• Prefill-Decode Disaggregation in Practice: Governing TTFT and TPOT for Long-Context LLM Serving

  A practical guide to Prefill-Decode disaggregation for LLM serving: principles, production architecture, monitoring, and a go-live checklist to govern TTFT and TPOT under long-context, high-concurrency workloads.

  06/30/2026 LLM Serving / Prefill Decode / Inference Optimization / KV Cache / SLO

• LLM Model Routing in Practice: Cost, Latency, and Quality Driven by Evaluation

  A practical guide to LLM model routing: signals, evaluation loops, cost budgets, fallback strategies, and go-live checks for dynamic model selection.

  06/30/2026 LLM / Model Routing / AI Gateway / Evaluation / Inference Cost