In Brief
Online action-stacking makes controller-style, burst commands possible and trains agents that use far fewer actions while matching the performance of much larger command sets.
ON THIS PAGE
Key Findings
Combining online action-stacking with a reward that encourages repeated (burst) actions produces sparse, realistic control behaviour for air traffic tasks. For lateral navigation, a 5-choice action setup with action-stacking matched the behaviour of a much larger 37-choice action setup, while issuing far fewer commands. Penalizing rapid action changes (action damping) reduced noisy oscillations and produced controller-style bursts rather than continuous tweaks. Tests used simple one- and two-aircraft scenarios but show this approach can keep safe separation and route/level tracking with fewer instructions. Guardrails Pattern Consensus-Based Decision Pattern
Data Highlights
1Baseline policy (no damping) averaged ~113 actions per episode over 100 episodes, showing frequent, noisy commands.
2Damped policy with action incentives averaged 14.5 actions per episode (standard deviation 6.6) in the two-aircraft lateral navigation tests.
3Comparable lateral-navigation performance achieved using a 5-dimensional action space instead of a 37-dimensional action space (matching task success while reducing action complexity).
What This Means
Engineers building automated air traffic decision support or autonomy should care because fewer, burst-style commands make agent behaviour easier to validate, align with human controller practice, and reduce action dimensionality for training. Technical leads and researchers can use this to speed up training and simplify verification when moving from simulation to pre-production tests. Safety and operations teams will benefit from agents that issue sparse, interpretable instructions compatible with human workflows. Supervisor Pattern
Explore evaluation patternsSee how to apply these findings
Key Figures
Fig 1: Figure 1 : The interaction loop of agent with environment in a Markov Decision Process.
Fig 2: Figure 2 : The “X-Plus Sector”, an artificial representation of en-route airspace with 20 nm airways and named navigation fixes.
Fig 3: Figure 3 : Viable routes through the X-Plus sector from each starting point.
Fig 4: Figure 4 : Policy controlling 2 aircraft through the X-Plus sector with no action damping penalty and no action stacking. A total of 92 separate actions were issued.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreLimitations
Results come from controlled, small-scale scenarios (one or two aircraft in an artificial sector), so performance on dense, real-world traffic or full multi-sector coordination is untested. The method depends on reward shaping (penalizing frequent actions and rewarding bursts), which needs careful tuning and may transfer poorly across different traffic mixes. Safety-critical deployment will require extensive validation beyond these simulated experiments and combination of lateral, vertical, and deconfliction controls remains future work. Chain of Thought Pattern
Methodology & More
Researchers trained agents in simple en-route scenarios targeting four controller behaviours: follow a route, reach the coordinated exit level, keep required separation, and minimise instructions. Scenarios were 300 simulation steps each (6-second steps), and training included one- and two-aircraft cases. The main change tested was online action-stacking: when the agent issues an instruction, that instruction can persist for a short burst of steps, letting the agent express compound commands using a much smaller set of discrete choices. A complementary reward term discouraged issuing actions every step and encouraged repeated bursts of the same command. Tree of Thoughts Pattern The combination produced much sparser and more controller-like behaviour. Without damping, policies constantly flipped commands and issued around 113 actions per episode. With action-damping and stacking, mean actions dropped to about 14.5 per episode while maintaining navigation success and separation. Crucially, lateral navigation performance that previously required a 37-choice action set could be matched with a 5-choice set when using online action-stacking, reducing action-space complexity and training cost. The approach looks promising for making learned controllers easier to inspect and closer to human practice, but it still needs scaling to realistic traffic densities and integration of lateral, vertical, and conflict-resolution tasks before operational use.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
University of Exeter authors with low h-indices and arXiv venue; recognized institution but limited author impact.