Distributed LLM Fine-Tuning Checkpoint Resume in Production: Sharded Checkpoints, Atomic Commits, and World Size Changes
A practical guide to production-grade checkpoint and resume for distributed LLM fine-tuning. Covers sharded checkpoints, atomic commits, optimizer/RNG state recovery, world size changes, and fault injection drills to prevent silent training drift.