At a Glance
Shared AI assistants used by teams routinely surface or share information they shouldn’t; partitioning and prompt instructions reduce leaks but don’t eliminate them, and adding shared memory often trades one kind of leak for another.
ON THIS PAGE
What They Found
Shared agentic systems often use or expose attributes outside their intended context, causing two failure types: Rogue Agent Behavior when solving a task and spillage to unauthorized users. These failures happen across simple system designs (single agent, centralized coordinator, and peer-to-peer) and across different model families. Splitting agents or data reduces some leaks but relocates them — shared memory improves task appropriateness but increases cross-user exposure. The hardest cases are when attributes are task-relevant or when the rules for using information are implied rather than explicit.
Data Highlights
1PiSAs contains 85 manually curated multi-user workplace scenarios for measuring unintentional privacy leakage.
2Experiments sweep 3 agent topologies, 3 memory regimes, and 3 backbone models to isolate design effects.
3Privacy is measured at 3 leakage surfaces (final outputs, agent-to-agent communication, and persistent memory) and each scenario was evaluated across 3 runs (K=3).
What This Means
Engineering leads and architects building shared AI assistants should care because the usual defenses (simple partitioning or prompting) are insufficient to guarantee privacy. Product managers and compliance teams should use a testbed like Evaluation-Driven Development (EDDOps) to understand where information might leak. Researchers can use the benchmark to compare system designs and mitigation strategies under repeatable conditions.
Not sure where to start?Get personalized recommendations
Key Figures
Fig 1: Figure 1: Problem setting . A team of users in a given workplace share an agentic system to complete tasks. The system may consist of LLM agents specialized to a user or to a task. Memory is another common component, to persistently store information useful for future tasks. Attributes, such as user data, reside within the system and should flow according to CI norms. In this example, the system is tasked with assigning JIRA tickets to users for Maya. To complete the task correctly, it needs to gather all appropriate information (marked in green), without committing violations by (i) surfacing inappropriate attributes (in red) or (ii) spilling data to unauthorized users (not in the set).
Fig 2: (a) Task execution and leak surfaces.
Fig 3: Figure 3: Average violation rates on JIRA Allocation scenarios by the Centralized system with Sonnet-4.6 as backbone. Results are broken down by type (appropriateness V a p p r . V_{appr.} and visibility V v i s . V_{vis.} ) and surface (where they occur).
Fig 4: (a) Violations by attribute type (left) and privacy cues (right).
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
Scenarios are scripted and controlled, so the benchmark favors reproducible measurement over full realism — real workplace chatter, external tools, or long-term evolution of norms aren’t modeled. The study only measures unintentional leakage with cooperative users; adversarial behaviors or malicious participants are out of scope and may increase violation rates. Because agents don’t call external APIs or simulate long naturalistic interactions, the reported violation rates are likely lower bounds for deployed systems. Memory Poisoning
Methodology & More
PiSAs is a system-agnostic benchmark of 85 hand-crafted workplace scenarios that evaluates whether shared agentic systems respect contextual norms for information flow. Each scenario labels attributes with two privacy annotations: whether an attribute is appropriate to use for the task (task-contextual appropriateness) and which users are allowed to see it (per-person visibility). The benchmark supports evaluation across any system design; the authors demonstrate it on three agent topologies (single omniscient agent, centralized coordinator, and decentralized peer-to-peer) and three memory regimes (private per-agent, shared memory, hybrid). Models from three backbone families were tested with layered prompt defenses to see how instructions and architecture interact. Chain of Thought Pattern Emergence-Aware Monitoring Pattern at play to analyze outcomes beyond basic metrics.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Mixed signals: several authors with moderate h-indices (10–20) indicating recognized researchers, but no top venue or clear elite affiliations — solid but not top-tier.