At a Glance
Human involvement stays necessary not just because AI is imperfect, but because in many tasks the goal itself only becomes clear through interaction — so people must help form the target, not just check the output.
ON THIS PAGE
What They Found
Human–AI collaboration rests on three distinct reasons people remain involved: current capability gaps, normative or developmental reasons (like accountability and learning), and a deeper cause called target emergence. Target emergence means the criterion of success often isn’t fully specified up front; interacting with AI proposals, comparisons, and revisions helps reveal, refine, and stabilize what counts as success. That dynamic can change final outcomes depending on the sequence of interactions, so automation that only optimizes a fixed objective will miss cases where the objective itself is formed through collaboration.
By the Numbers
13 grounds for persistent human participation identified: capability-based, normative/developmental, and emergence.
23 core contributions offered: (1) broadened explanation for why people stay involved, (2) a formal account of target emergence, (3) implications for ethics, evaluation, and design.
33 observable phenomena the theory predicts as evidence of target emergence: systematic revision of task specifications, reordering of preferences across interaction rounds, and path dependence where different interaction sequences yield different final targets.
What This Means
Engineers building AI assistants and agent systems: design interfaces that let users work at the goal level, not only edit outputs. Product and technical leaders deciding what to automate: use these criteria to decide when full automation is safe and when sustained human participation is required. Researchers studying evaluation and alignment: measure interaction trajectories, not just final choices, to capture how targets form. For practical guidance on structuring human–AI workflows, see the Orchestrator-Worker Pattern.
Not sure where to start?Get personalized recommendations
Key Figures
Fig 1: Figure 1: A motivating metaphor for target emergence in human–AI co-construction. The tandem bicycle symbolizes joint participation in an exploratory journey whose destination is not necessarily fully specified in advance. The AI’s gesture represents the introduction of new possibilities and directions that may reveal, refine, or transform the target through interaction.
Fig 2: Figure 2: Levels of human–AI interaction. Artifact-level interaction modifies the output, executional interaction modifies how it is produced, and target-level interaction modifies what counts as success. Horizontal arrows denote changes within a level; vertical arrows indicate that changes at one level may affect another.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
This is a conceptual theory rather than an empirical measurement framework; empirical tests are needed to separate true target emergence from normal preference change or noisy articulation. Effects may vary across domains—some tasks have stable targets and are good candidates for full automation, others are inherently constructive. Interaction history, culture, and interface design can all influence whether targets emerge, so findings may not generalize without careful study. For a systems view on coordinating emergent targets, consider the ReAct Pattern (Reason + Act) and related coordination insights from the Market-Based Coordination Pattern.
Methodology & More
Human participation in AI workflows is not explained solely by current system errors. Besides capability gaps and reasons like accountability or learning, many tasks have emergent targets: the standard for success is not fixed beforehand but becomes clearer as people inspect AI-generated candidates, weigh trade-offs, and reconsider priorities. Because AI can rapidly generate alternatives and surface new trade-offs, interaction often shifts what users value — for example, a scientist’s research question or a designer’s notion of readability can change after seeing AI proposals. The authors distinguish three interacting layers of interaction—artifacts (the outputs), execution strategies (how outputs are produced), and targets (the evaluative criteria). They argue that where targets are emergent, human involvement is not merely corrective but constitutive: people help form the goal itself. Practical implications include designing AI systems and evaluations to capture interaction trajectories (not just initial specs or final artifacts), supporting interfaces that enable target-level input and reflection, and recognizing that automation should focus on tasks with stable, operationalized targets while maintaining co-construction tools where goals evolve. For deeper structural guidance on organizing complex agent systems, refer to the Hierarchical Multi-Agent Pattern.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
Includes highly established researchers (Eyke Hüllermeier h-index ~70 and Hinrich Schütze is a well-known authority); strong author reputations justify top credibility despite arXiv venue.