Tag: Cost Governance

2 articles

• LLM Batch Inference Production Practices: Avoiding Duplicate Charges and Missing Results with Manifest, Idempotent Record IDs, and Shard Retry

  A systematic guide to building a recoverable, auditable batch inference pipeline for asynchronous large-scale LLM calls using job manifests, deterministic record IDs, per-record reconciliation, and shard-based retries to prevent duplicate billing and missing results.

  07/28/2026 LLM batch inference / idempotency / job orchestration / reliability engineering / cost governance / shard retry

• LLM Tokenizer Version Governance in Production: Preventing Cost Surprises with Compatibility Fingerprints, Token Drift Replay, and Budget Gates

  Tokenizer changes can silently shift token counts, truncation boundaries, and costs for the same prompt. This article provides practical solutions including compatibility fingerprints, golden corpus replay, dual counting, and budget gates to help teams identify risks before model upgrades and safely roll out changes.

  07/11/2026 LLM Engineering / Tokenizer / Model Migration / Cost Governance