Tag: CUDA

3 articles

• LLM Inference Reproducibility in Production: Controlling Output Drift with Seed Contracts, Batch Invariance, and Deterministic Kernels

  Same model, prompt, and seed can still yield different outputs. This article covers sampling state, dynamic batching, floating-point reduction, parallel topology, and version fingerprints to provide tiered reproducibility goals, implementation strategies, replay gates, and a deployment checklist.

  07/15/2026 LLM Serving / Reproducibility / vLLM / CUDA / MLOps / Batch Invariance / Deterministic Inference / Seed Contract

• LLM CPU–GPU Data Path in Production: Reducing Inference Jitter with NUMA Affinity, Pinned Memory, and Async Copy

  A systematic guide to optimizing the CPU-to-GPU data path for LLM inference, covering NUMA affinity, pinned memory, async copy, topology scheduling, and container alignment to reduce host-side transfer jitter, tail latency, and cross-node resource misconfiguration.

  07/12/2026 LLM Serving / CUDA / NUMA / GPU Performance / PyTorch

• LLM GPU Memory Fragmentation in Production: Taming OOMs with Memory Snapshots, Allocator Tuning, and Admission Control

  LLM inference services often hit OOM due to fragmentation, not total capacity exhaustion. This guide covers Memory Snapshots, dynamic shapes, allocator tuning, and vLLM memory budgeting for observable, verifiable, and rollback-safe production management.

  07/11/2026 LLM Serving / CUDA / GPU Memory / PyTorch / vLLM