At a Glance
People choose to observe others when simulating their goals and beliefs promises more useful information than acting now; a model that combines mind-reading with cost–benefit reasoning predicts human observation choices closely.
ON THIS PAGE
What They Found
Humans balance the benefit of watching another agent (what that agent’s actions reveal about the world) against the cost of delaying their own actions. A model that infers others’ goals and beliefs and then compares the expected utility of observing versus acting reproduces human observation behavior across four increasingly complex multi-agent tasks. Models that either lack the mentalizing step or lack the cost–benefit comparison fail in predictable ways: one over-observes, another under-observes, and a naive policy cannot pick whom to watch. People still hedge slightly more than the rational model predicts, suggesting curiosity or uncertainty about others’ models.
Not sure where to start?Get personalized recommendations
Data Highlights
1Rational mentalizing model pooled agreement with human observation counts: Lin’s concordance correlation coefficient = 0.89 (95% CI [0.84, 0.92]).
2Per-experiment correlation between model and humans: r between 0.90 and 0.95 across the four experiments.
3Human split-half reliability (upper bound for model fit) ranged from 0.83 to 0.92, showing the model approaches the ceiling of predictable behavior.
What This Means
Engineers building multi-agent systems and developers of agent monitoring or delegation tools can use this to decide when an agent should observe peers versus act independently. Research teams designing human-like decision policies or trust signals can use the model to predict and explain selective observation and to calibrate when to surface other agents’ behavior to users.
Key Figures
Fig 1: Figure 1 : Illustration of the social learning game. The participant controls the red (main) agent and must retrieve a target treasure chest. On each timestep, the participant can either observe a selected NPC (causing only that NPC to move) or act (moving the red agent one step). The map contains wizards (only one of each color holds the amulet), colored barriers, and treasure chests. The legend shows the main agent, expert NPCs, wizards, barriers, and treasure chests.
Fig 2: Figure 2 : Schematic of the Rational Mentalizing model. At each timestep, the observer (red agent) decides whether to act or observe another agent. Utility comparison: it weights the cost of acting now ( C self C_{\text{self}} ) against the expected cost of watching the expert for another step ( Q obs Q_{\text{obs}} ), and observes only when watching is cheaper. Theory of Mind: to establish that cost, the observer infers the observed agent’s goal and belief, for example whether it is heading to treasure A or B, and predicts how much observing would reduce its own uncertainty.
Fig 3: Figure 3 : Observation steps pooled across all four experiments. Each panel shows one model, with points colored by experiment. Pearson r r and Lin’s concordance correlation coefficient (CCC) with 95% bootstrap confidence intervals are shown. The Rational Mentalizing model achieves the highest agreement with human observation behavior (CCC = 0.89 =0.89 ), while the alternative models show low concordance, indicating poor absolute agreement.
Fig 4: Figure 4 : Qualitative examples of observation behavior across the four experiments. Each panel shows a trial map with agent trajectories and a bar chart comparing the number of observation steps for humans (H) and four models: Rational Mentalizing (RM), Mentalizing Observer (SM), Rational Observer (RN), and Naive Observer (N). Panels A.1–A.2: Experiment 1 (single agent, single goal). Panels B.1–B.2: Experiment 2 (single agent, multiple goals). Panels C.1–C.2: Experiment 3 (two expert agents, multiple goals), with separate observation counts for Agent 2 (blue) and Agent 3 (green). Panels D.1–D.2: Experiment 4 (expert and novice agents, multiple goals). Error bars indicate standard error across participants.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
The model treats observation as the only social channel and does not model explicit teaching, communication, or persuasive intent, which can change the value of watching. Trials were independent and did not let observers accumulate long-term reputations for specific agents, so the findings may differ when history-based trust matters. Across all tasks the model underpredicts how much people watch, suggesting additional drivers like curiosity, risk aversion, or uncertainty about the model of others are at play.
Methodology & More
Participants played a grid-based treasure task where their avatar could either move toward a chest or spend a step observing another agent’s movement. Each map contained wizards (only one holds a needed amulet), colored barriers, and multiple treasure chests; observing an agent reveals how that agent plans to get to its chest and which wizard holds the amulet. Four experiments varied goal ambiguity, the number of observable agents, and agent expertise (expert vs novice) so that the trade-offs between social observation and solo exploration could be tested systematically.
The Rational Mentalizing model builds a short mental simulator of each observed agent (inferring likely goals and beliefs), predicts how watching would narrow the observer’s uncertainty, and then compares the expected cost of continued observation (including time cost) against the cost of committing and acting now. Compared to three ablated models that removed either the mentalizing inference, the utility comparison, or both, the full model closely matched human observation counts (pooled concordance = 0.89) and reproduced qualitative patterns such as choosing which of two agents to watch and shifting attention depending on agent expertise. Ablations failed in predictable ways: lacking cost comparison led to over-observing, lacking mentalizing led to under-observing or indiscriminate watching, and naive policies could not choose whom to watch. The model generally reproduced relative patterns across conditions but systematically underpredicted absolute observation rates, pointing to extra motives like curiosity or conservative evidence-gathering in people.
Implications: for multi-agent system design and agent-to-agent evaluation, equip observers with a simple mental model of peers and a cost–benefit rule to decide when to monitor others. That combination yields humanlike selective observation and can improve when and whom to consult in delegation, monitoring, or trust-calibration scenarios. Future work should add communication channels, accumulated reputations, and intrinsic information-seeking to close the remaining gap between the rational account and human behavior.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Mixed signals: includes recognizable researchers (e.g., Joshua B. Tenenbaum and Samuel J. Gershman) but provided h-index values are modest and the paper is an arXiv preprint with no listed affiliations or citations — solid but not top-tier under the rubric.