When Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents

Source: arXiv cs.CR

Published: Thu, 09 Jul 2026 00:00:00 -0400

Summary

arXiv:2607.06595v1 Announce Type: new

Abstract: Personal AI agents powered by large language models can reason and act using available tools to access emails, manage calendars, and push code to remote repositories, all with minimal oversight. When augmented with long-term memory, an agent can recall specific details relevant to the current task, reducing the need for large context windows. Currently, long-term memory agents tend to fall into two distinct domains: conversational and action-planning agents. Personal assistant agents sit at the convergence of these two domains and handle sensitive information while interacting with untrusted information sources, creating previously unaccounted security vulnerabilities. In this work, we introduce the novel attack vector, GhostWriter, which exploits current memory subsystems in tool-using personal agents to poison their memory store. GhostWriter operates in two phases: injection, where an adversary sends a hidden attack payload to the target agent; and activation, in which the poisoned memory is retrieved. We show that GhostWriter achieves near-universal injection rates of approximately 98% and a high average activation rate of approxi

Sources

• Original publication on arXiv cs.CR
• arXiv cs.CR feed
• Lyrie research desk