Explainability builds trust and enables debugging. Users and operators need to understand agent reasoning.
Levels
- What: Describe the output
- How: Show the process
- Why: Explain the reasoning
Techniques
- Attention visualization
- Chain-of-thought logging
- Counterfactual analysis
- Feature importance