Tag: Agent

3 articles

• LLM Real-Time Voice Input in Production: Low-Latency Recognition with VAD Segmentation, Partial/Final Reconciliation, and Audio Contracts

  Real-time voice applications face challenges beyond model accuracy. This article systematically breaks down how to build an observable, rollback-capable low-latency speech recognition pipeline—from audio format contracts and VAD segmentation strategies to Partial/Final state reconciliation, timestamp replay, and real-world noise evaluation. Designed for LLM, generative AI, and real-time Agent practitioners.

  07/13/2026 LLM / real-time voice / speech recognition / VAD / production engineering / Agent / RAG

• LLM-as-a-Judge in Production: Rubrics, Biases, and a Closed-Loop Regression Testing System

  A practical guide to deploying LLM-as-a-Judge in production for RAG, Agent, and content generation systems, covering scoring modes, rubric design, bias sources, online regression, human review, and threshold governance.

  06/30/2026 LLM Evaluation / LLM-as-a-Judge / RAG / Agent / AI Engineering

• Prefill-Decode Disaggregation in Practice: Taming Latency for Long-Context LLM Serving

  Long contexts and agentic requests shift LLM inference bottlenecks from single-card throughput to tail-latency governance. This post dives into Prefill-Decode disaggregation, KV cache transfer, resource planning, and a pre-launch checklist to move LLM serving from 'working' to stable and scalable.

  06/29/2026 LLM Serving / Prefill Decode / KV Cache / Inference Optimization / Long Context / Tail Latency / RAG / Agent