Why can't I just overwrite the original vectors when upgrading an embedding model?

Old and new embeddings often differ in dimensionality, distance metrics, and semantic spaces. Directly overwriting mixes old queries, new documents, and cached results, causing recall drift and making rollback nearly impossible.

Should I use dual indexes or named vectors?

Dual indexes work for any vector store and large-span migrations. Named vectors are simpler for collections already supporting multiple vector fields, offering easier rollback but requiring explicit vector name switching in queries.

Can a recall gate replace business evaluation?

No. A recall gate only confirms the ANN index hasn't degraded significantly. You still need golden query sets, human annotations, or shadow traffic to validate final business relevance.

LLM Embedding Index Migration in Production: Avoiding Recall Drift with Dual-Writes, Alias Switching, and Recall Gates

Background: Embedding Is Not a Regular Field — Upgrades Rewrite the Recall Space

In LLM applications, the embedding index typically serves as the underlying infrastructure for RAG, semantic search, recommendation recall, or knowledge base Q&A. The problem is that upgrading an embedding model changes more than just an API parameter — it alters the entire vector space.

A seemingly simple upgrade can simultaneously change four things:

Change Dimension	Specific Impact
Vector Dimensionality	`text-embedding-3-small` defaults to 1536 dimensions, `text-embedding-3-large` defaults to 3072
Distance Metric	Cosine, Euclidean, and Dot Product are not interchangeable
Text Truncation Strategy	New models may use different tokenizers or truncation logic
Semantic Similarity Distribution	The same query-doc pair scores significantly differently under old vs. new models

In OpenAI’s documentation, text-embedding-3-small outputs 1536 dimensions by default, while text-embedding-3-large outputs 3072, and both support shortening vectors via the dimensions parameter. Similarly, Pinecone requires the index’s dimension and similarity metric to match the embedding model when creating a dense vector index. This demonstrates that the index structure is tightly coupled with the embedding model and should not be mixed arbitrarily in production.

A more insidious risk is recall drift. Even if the new model scores higher on offline benchmarks, the top-k results for online queries can change significantly. For scenarios like customer service knowledge bases, compliance Q&A, and enterprise search, changes in recall results directly impact answer stability. Therefore, the focus of embedding index migration is not just “re-running a vectorization script,” but building a verifiable, switchable, and rollback-capable release process.

Core Principle: Treat Index Migration as a Data Plane Release

Production-grade index migration can be broken down into three planes:

Data Plane: Source documents, chunks, metadata, embedding vectors, index shards, and deletion events must remain consistent. Migrating only vectors while ignoring document versions leads to vectors pointing to outdated text.
Query Plane: Query requests must know which embedding model, index version, and distance metric are currently in use. Otherwise, query vectors from the new model will land on the old index, distorting results.
Release Plane: Migration must support background backfill, incremental dual-writes, validation gates, traffic switching, and fast rollback.

Qdrant’s official documentation on embedding model migration outlines two typical approaches: one is blue-green migration, which involves creating a parallel collection, dual-writing to both old and new collections, re-embedding in the background, and then switching search traffic upon completion. The other is adding a new model vector field to a collection that supports named vectors, backfilling, then switching the query’s using parameter, while retaining the old vectors for rollback.

The common principle behind both approaches is: never perform destructive in-place updates on the old index. During migration, both old and new retrieval paths must be able to coexist.

Recommended Architecture: Dual-Writes, Backfill, Alias Switching, Gate Validation

1. Establish Index Versioning

Don’t just name your indexes docs, faq, or kb_vectors. Production environments should explicitly record the index version:

index_profile:
  logical_name: kb-search
  physical_index: kb-search-v20260708-embed3-small
  embedding_model: text-embedding-3-small
  dimension: 1536
  metric: cosine
  chunker_version: chunker-v4
  metadata_schema: kb-meta-v3
  status: building

logical_name is used by the business logic, physical_index is the actual vector store index, and chunker_version and metadata_schema help debug recall changes. Many recall issues are not caused by the embedding model itself, but by simultaneous changes in chunking rules, cleaning rules, or metadata filters.

2. Create a New Index, Don’t Overwrite In-Place

When the new model’s dimensions or metric change, creating a new physical index is the safer choice. Pinecone’s index creation documentation explicitly requires specifying dimension and similarity metric for dense vector indexes, matching the embedding model. Qdrant’s collections also require consistent dimensions and metrics for vectors within the same collection (unless using named vectors).

If your vector store supports collection aliases or index aliases, you can let the business logic always access the logical name:

Qdrant: Collection aliases allow atomically switching queries from one collection to another
Milvus: Aliases decouple application code from specific physical collections, supporting zero-downtime data updates, A/B testing, and blue-green deployments

3. Incremental Dual-Writes to Avoid Data Loss During Backfill

The most error-prone part of migration is incremental data. The correct approach is to put the write path into dual-write mode:

async function upsertDocument(doc: SourceDocument) {
  const oldVector = await embed(doc.text, "old-embedding-model");
  const newVector = await embed(doc.text, "new-embedding-model");

  await Promise.all([
    vectorStore.upsert("kb-search-v1", doc.id, oldVector, doc.metadata),
    vectorStore.upsert("kb-search-v2", doc.id, newVector, doc.metadata),
  ]);

  await ledger.record({
    doc_id: doc.id,
    old_index: "kb-search-v1",
    new_index: "kb-search-v2",
    source_hash: sha256(doc.text),
    migrated_at: new Date().toISOString(),
  });
}

Dual-writes are not just calling upsert twice; you must also record a request ledger: source document hash, chunk ID, embedding model, index version, write status, and failure reason. Without a ledger, you can only guess which data was missed later.

4. Background Backfill of Historical Data

For backfilling existing data, the recommended process is:

Scan source data → Re-chunk → Generate new embeddings → Write to new index → Validate counts and hashes

Do not try to derive new vectors from old ones, and do not simply copy old index payloads and declare migration complete.

The backfill process must support resumable execution:

CREATE TABLE embedding_migration_task (
  id BIGINT PRIMARY KEY,
  doc_id VARCHAR(128) NOT NULL,
  chunk_id VARCHAR(128) NOT NULL,
  source_hash VARCHAR(64) NOT NULL,
  target_index VARCHAR(128) NOT NULL,
  status VARCHAR(32) NOT NULL,
  retry_count INT NOT NULL DEFAULT 0,
  last_error TEXT,
  updated_at TIMESTAMP NOT NULL
);

The core purpose of this table is to answer three pre-release questions:

How many chunks are left to migrate?
Are failures concentrated on a certain type of document?
Is there a discrepancy in the number of chunks for the same doc between old and new indexes?

Recall Gates: Validate the Index First, Then Validate the Business

Index migration requires at least two layers of gates.

Layer 1: ANN Recall Gate

Qdrant’s ANN recall documentation defines recall@k as how closely approximate nearest-neighbor search matches exact kNN search, and recommends calculating recall@k in CI using exact-search mode, blocking the release if it falls below a threshold. This layer primarily checks:

Index construction parameters (HNSW’s M, efConstruction)
Search parameters (ef)
Shard status
Vector write quality

Layer 2: Business Relevance Gate

This requires maintaining a set of golden queries: common user questions, long-tail questions, compliance-sensitive questions, easily confused questions, and historically failed online queries. For each query, hit both the old and new indexes and compare:

Top-k document overlap rate
Target document hit rate
Whether answer citations change
Whether human-annotated relevance decreases

An executable gate configuration:

release_gate:
  ann_recall_at_10_min: 0.95
  migrated_chunk_ratio_min: 0.999
  failed_chunk_count_max: 0
  golden_query_target_hit_rate_drop_max: 0.02
  no_unknown_schema_version: true
  rollback_ready: true

ANN recall only confirms that the approximate index hasn’t deviated significantly from exact search; it doesn’t mean the new embedding model better understands business intent. Business gates still require query-level evaluation.

Switching Strategy: Aliases First, Configuration as Fallback

Once all gates pass, you can proceed to small-scale traffic switching. It’s better to bind your business code to a logical alias rather than a physical index name.

With Qdrant, for example, collection aliases allow the application to continue accessing production_collection, while the backend atomically switches the alias from the old collection to the new one — alias actions are atomic and won’t affect concurrent requests. Milvus’s aliases provide a similar abstraction.

During switching, it’s recommended to keep three types of switches:

search_runtime:
  active_index_alias: kb-search-prod
  active_embedding_model: text-embedding-3-small
  shadow_index: kb-search-v2
  shadow_sample_rate: 0.05
  rollback_index: kb-search-v1

During the canary phase, the main online results still come from the old index, while a portion of requests are shadowed to the new index and differences are recorded. After confirming the differences are manageable, switch the alias. Don’t delete the old index immediately after switching; keep at least an observation window.

Rollback Strategy: Roll Back the Query Path, Not Re-Migrate

Rollback must be prepared before going live. A common mistake is waiting until the new index has issues before starting to rebuild the old one. The correct approach is to keep the old physical index, the old embedding query path, and the old configuration version.

When rolling back, prioritize three things:

Point the query alias back to the old index
Restore the old embedding model for query embedding
Stop dual-writes to the new index or switch to asynchronous compensation

As long as the old index remains incrementally synchronized during migration, rollback is just a configuration switch, not a data recovery operation.

For the named vectors approach, rollback is even more direct: the old vector field is still in the same collection, you just need to switch the query’s using parameter back to the old vector name.

Applicable Scenarios

Scenario	Description
Embedding Model Upgrade	Upgrading from an old version to a new one
Vector Dimensionality Change	Switching from 1536 to 1024 or 3072
Hybrid Search Upgrade	Upgrading from single dense vector to dense + sparse hybrid
Chunk Rule Change	Rebuilding index due to chunking strategy adjustments
Vector Store Migration	Migrating from one cluster to another
High Recall Stability Requirements	Cannot tolerate “random search quality fluctuations after update”

If it’s just a small number of offline experiments, or the index has no online traffic, you can simplify to a one-time rebuild. But as soon as it’s serving production search, it should be treated as a data release process.

Common Pitfalls

Pitfall 1: Only Comparing Final Answers, Ignoring Recall

When answer quality drops, the problem could come from the generation model, prompt, reranker, or recall. During index migration, you must isolate the recall layer for validation first; otherwise, troubleshooting becomes very difficult.

Pitfall 2: Mixing Old and New Embeddings in the Same Field

Mixing vectors generated by different models in the same vector field is usually a catastrophic mistake. It makes distance scores uninterpretable and removes the boundary for rollback.

Pitfall 3: Ignoring Deletions and Partial Updates

Qdrant’s blue-green migration guide also warns that deletions or partial updates during migration need to be paused or handled with extra logic — because background backfill might re-write already deleted data into the new index.

Pitfall 4: Cleaning Up the Old Index Immediately After Switching

The old index is your rollback insurance. Don’t rush to release it until the observation window is over, difference monitoring is stable, and the business confirms the new results are acceptable.

Pre-Release Checklist

Before going live, at least confirm the following: