Emergence is both exciting (new capabilities) and concerning (unexpected behaviors) in AI systems.
Examples
- Chain-of-thought reasoning
- In-context learning
- Code generation
- Multi-step planning
Implications
- Hard to predict capabilities
- May include undesired behaviors
- Requires empirical discovery
- Complicates safety analysis