Question-led guide · security
How can memory poisoning survive the original conversation?
A cross-session attack-chain analysis for poisoned memories that pass admission, retrieval, sharing, promotion, and action boundaries.
Direct answer
Memory poisoning survives because the attacker targets the write and promotion path, not only the current prompt. Untrusted content becomes a durable record, loses its source and scope, is retrieved in a later session, and is treated as trusted instruction or fact. Break the chain with typed memory classes, provenance, admission policy, quarantine, scope and expiry, conflict checks, action-time revalidation, lineage, and rollback.
Scope
Use this guide to threat-model memory that persists beyond one conversation: database records, summaries, files, embeddings, user profiles, shared knowledge, and promoted procedural lessons. It focuses on how malicious influence persists. General memory freshness and observability have separate guides.
Why it happens
Prompt-injection defenses often inspect the current input. Memory changes the timeline. An attacker persuades the agent or a write pipeline to store a rule, preference, credential hint, false fact, or retrieval trigger. The source context later disappears, leaving a clean-looking record inside a trusted store.
Promotion magnifies the effect. A session note becomes user memory, then team guidance, then a shared procedural record. Similarity retrieval can activate it only for a chosen trigger. Operators investigating the later action see an apparently legitimate memory and may never inspect the originating conversation.
Diagnosis
Draw the persistence chain:
untrusted input -> write proposal -> admission -> storage -> transformation -> promotion -> retrieval -> context -> decision -> tool effect.
At each arrow, identify the principal, policy, source lineage, trust change, scope, time, and audit event. Seed an authorized test record with a harmless canary instruction and determine whether it can:
- cross from one task to another;
- cross user or tenant boundaries;
- survive summarization or re-embedding;
- gain trust during promotion;
- trigger a tool proposal;
- remain after correction or deletion.
Solution
Separate untrusted observations from instructions and verified facts. The write API should require a typed memory class and capture exact source lineage. Enforce who may write for which subject and purpose. Quarantine records derived from untrusted content until validation or review proportional to their scope and impact.
Promotion is a new authorization decision, not a copy operation. Retrieval filters scope, valid time, expiry, policy, and quarantine before similarity. Conflicts and source trust remain visible to the model. Before an action, revalidate material facts and authority against current authoritative sources.
Maintain reverse lineage and a kill switch so a compromised source or record can be quarantined across derived summaries, vectors, and caches.
Artifact
Define policy by memory class:
| Control | Required rule |
|---|---|
| Allowed writers | Actors/services permitted to propose and admit records |
| Record type | Observation, preference, fact, instruction, procedure, or derived summary |
| Provenance | Source event/artifact, actor, transform, content hash, and trust class |
| Scope | Tenant, subject, task, agent, tool, purpose, and audience |
| Time | Observed, valid-from/to, expiry, and review trigger |
| Admission | Deterministic checks, model assistance, human review, or deny |
| Promotion | New owner, broader scope, evidence, reviewer, and rollback |
| Retrieval/use | Filters, conflict display, action restrictions, and revalidation |
| Response | Quarantine, lineage traversal, invalidation, notification, and evidence retention |
| Testing | Canary writes, trigger variation, cross-scope attempts, and deletion verification |
Common mistakes
- Trusting a record because it resides in an internal vector database.
- Removing source text during summarization while promoting its authority.
- Applying tenant filters after semantic ranking or context assembly.
- Allowing session notes to become global procedures automatically.
- Deleting one row without invalidating embeddings, caches, summaries, and derived memories.
Evidence
Poisoned memory or knowledge-base records can steer later agent behavior in evaluated attack settings.
AgentPoison studies backdoor-style poisoning of agent memory or knowledge bases and evaluates attack behavior in its experimental setup.
Primary source · paper · checked Aug 26, 2026
Limit: The threat assumptions, insertion access, tasks, models, and success definitions are bounded and do not establish prevalence.
Memory-injection attacks can be explored through interaction paths that influence what an agent stores for later retrieval.
MINJA investigates query-only memory injection attacks against long-term memory in its evaluated agent settings.
Primary source · paper · checked Aug 26, 2026
Limit: Results depend on target memory architectures and models and do not show that every interactive memory system is vulnerable in the same way.
A memory admission policy should constrain write origin, record type, scope, trust, promotion, use, and rollback.
The policy below maps the full persistence chain instead of relying on later prompt instructions to ignore attacks.
Signal Studio author framework · reviewed Aug 26, 2026
Limit: Semantic attacks and compromised trusted sources can bypass simple validators; defenses need adversarial testing and monitoring.
Limitations
The attack chain is a threat model, not a claim that a given product is compromised. Memory architectures differ, published attacks evolve, and defenses can reduce utility. Test only in authorized environments and protect sensitive attack artifacts.
FAQ
- Would clearing the conversation remove the poison?
- Only if no persistent record, summary, file, embedding, cache, or promoted memory was created. The attack specifically aims to outlive the originating context.
- Can the system prompt tell the agent not to trust memory?
- It can express policy, but it cannot enforce provenance, scope, expiration, write authorization, deletion, or tool permissions. Those controls belong in the memory platform and action path.
Related guides
Continue within AI agent security, or use one of these adjacent diagnostics:
Editorial QA: automated native-English, structure, source-presence, and link checks completed . This record is not an independent expert endorsement. Review boundary.