How should retrieval preserve provenance?

Scope

Use this schema for retrieval-augmented systems that ingest documents or records, transform them into an index, select evidence, and generate claims or actions. It supports debugging, citation, audit, correction, and deletion. It does not claim that a provenance graph is a truth graph.

Why it happens

RAG pipelines commonly retain a URL and chunk text. That loses the source version, parser, page or field, neighboring context, embedding model, filters, ranking scores, query rewrite, and exact excerpt placed in the prompt. When an answer is challenged, the team cannot tell whether the source changed, the parser dropped a qualifier, the retriever selected the wrong version, or the model exceeded the passage.

Provenance also has two directions. “Where” identifies the origin of a statement. “Why” records why this source and chunk contributed to this run: query, filters, ranker, score, policy, and selection decision.

Diagnosis

Take one generated claim and attempt a reverse trace:

claim -> context evidence unit -> retrieval candidate -> index record -> chunk -> parsed artifact -> captured source.

At each edge, verify stable ID, version, timestamp, transformation, actor or service, and integrity reference. Then replay forward using the saved query, filters, index snapshot, and ranker. If the same evidence set cannot be approximated or the system cannot explain divergence, mark the run non-reproducible.

Check whether citations support the exact claim, including qualifiers. A citation that merely shares the topic is a provenance link without evidentiary support.

Solution

Assign immutable IDs at capture and every material transformation. Keep content hashes where lawful. A chunk record should point to its source artifact and exact structural location; an index record should point to the chunk and embedding configuration; a retrieval run should retain query versions, scope filters, candidates, scores, reranking, and exclusions.

When compiling context, create evidence units containing the exact text, source identity, version, location, authority and freshness labels, and limitations. The generator should map each material claim to one or more evidence-unit IDs. Store the citation actually shown to the user and any downstream decision or effect that relied on the claim.

Artifact

A minimal evidence-supply-chain record:

source: {id: src-42, version: sha256:..., url: ..., captured_at: ...}
transform: {parser: parser-3.2, artifact_id: art-91, location: "section-4/table-2"}
chunk: {id: chunk-188, strategy: semantic-v5, parent: art-91}
index: {id: idx-12, embedding: embed-model@version, indexed_at: ...}
retrieval: {run: ret-731, query_version: q-8, scope: tenant-a, candidates: [...], selected: [...]}
context_unit: {id: ev-17, exact_text_hash: ..., placement: 6, limitations: [...]}
claim: {id: claim-9, text_hash: ..., supported_by: [ev-17], reviewer: ...}
effect: {id: effect-3, authorized_by: ..., based_on: [claim-9]}

Keep sensitive query and tenant fields under the same access and retention policy as the underlying content.

Common mistakes

• Storing a mutable URL with no capture or content version.
• Recording selected chunks but not excluded high-ranking candidates or scope filters.
• Citing a whole document when a transformed excerpt omitted its qualifier.
• Assuming content hashes establish authority or truth.
• Building lineage that cannot propagate correction, quarantine, or deletion.