Tag: GPU Memory

2 articles

• LLM Activation Checkpointing in Production: Selective Recompute, Non-Reentrant Mode, and RNG Consistency to Reduce Training Memory

  During large model training, activations often exhaust GPU memory before parameters. This article explores selective recomputation, non-reentrant mode, RNG state consistency, and engineering gating strategies to reduce peak memory while preserving gradient correctness and controlling throughput loss.

  07/14/2026 LLM Training / Activation Checkpointing / PyTorch / GPU Memory / Distributed Training

• LLM GPU Memory Fragmentation in Production: Taming OOMs with Memory Snapshots, Allocator Tuning, and Admission Control

  LLM inference services often hit OOM due to fragmentation, not total capacity exhaustion. This guide covers Memory Snapshots, dynamic shapes, allocator tuning, and vLLM memory budgeting for observable, verifiable, and rollback-safe production management.

  07/11/2026 LLM Serving / CUDA / GPU Memory / PyTorch / vLLM