Tag: Agent

2 articles

• LLM-as-a-Judge in Production: Rubrics, Biases, and a Closed-Loop Regression Testing System

  A practical guide to deploying LLM-as-a-Judge in production for RAG, Agent, and content generation systems, covering scoring modes, rubric design, bias sources, online regression, human review, and threshold governance.

  06/30/2026 LLM Evaluation / LLM-as-a-Judge / RAG / Agent / AI Engineering

• Prefill-Decode Disaggregation in Practice: Taming Latency for Long-Context LLM Serving

  Long contexts and agentic requests shift LLM inference bottlenecks from single-card throughput to tail-latency governance. This post dives into Prefill-Decode disaggregation, KV cache transfer, resource planning, and a pre-launch checklist to move LLM serving from 'working' to stable and scalable.

  06/29/2026 LLM Serving / Prefill Decode / KV Cache / Inference Optimization / Long Context / Tail Latency / RAG / Agent