LLM CPU–GPU Data Path in Production: Reducing Inference Jitter with NUMA Affinity, Pinned Memory, and Async Copy
A systematic guide to optimizing the CPU-to-GPU data path for LLM inference, covering NUMA affinity, pinned memory, async copy, topology scheduling, and container alignment to reduce host-side transfer jitter, tail latency, and cross-node resource misconfiguration.