Tag: FlashAttention

2 articles

• LLM SFT Sequence Packing in Production: Boosting Throughput with Sample Boundaries, Loss Mask, and Length Buckets

  A practical guide to implementing safe Sequence Packing for SFT training, covering attention isolation, loss masking, position IDs, length bucketing, and validation gates to eliminate padding waste without cross-sample contamination.

  07/14/2026 LLM Training / SFT / Sequence Packing / FlashAttention / Data Collator

• LLM Attention Backend in Production: Managing FlashAttention and FlashInfer with Compatibility Matrices, Numerical Replay, and Automatic Fallback

  LLM inference frameworks select kernels among FlashAttention, FlashInfer, and SDPA. This article provides a compatibility matrix, numerical replay, performance baselines, and automatic fallback strategies to prevent startup failures, output drift, and tail latency degradation after upgrades.

  07/12/2026 LLM Serving / FlashAttention / FlashInfer / GPU Kernel / Inference Engineering