Tag: Capacity Planning

2 articles

• LLM Capacity Load Testing in Production: Finding Real Safe QPS with Timed Traces, Burstiness, and Tail-SLO Gates

  LLM capacity testing shouldn't rely on fixed concurrency and average latency alone. This article combines vLLM, AIPerf, and MLPerf to explain Timed Traces, burst traffic, tail-latency gates, and Safe QPS — a production-grade method for finding the true capacity inflection point before launch.

  09/02/2026 LLM Serving / Capacity Testing / Benchmark / Tail Latency / Safe QPS / Timed Trace / Burstiness / Capacity Planning

• LLM Inference Capacity Planning in Production: Estimating GPU Headroom with Token Throughput Stress Tests and Queue Depth

  A practical guide to transforming LLM inference capacity from reactive scaling to measurable, estimable, and alertable capacity engineering. Covers token throughput, queue depth, P99 TTFT, GPU headroom, and a pre-launch checklist to help teams identify bottlenecks and make quantitative scaling decisions.

  07/05/2026 LLM Serving / Capacity Planning / GPU Inference / SLO / vLLM / Token Throughput