The EU AI Act requires high-risk AI systems to have a human oversight mechanism capable of overriding or stopping the system. But the Act is silent on who bears liability when the shutdown mechanism fails to trigger.
Key Analysis
EU AI Act Article 9 requires a risk management system including the ability to stop the AI system.
Liability for a failed shutdown gate may rest with the developer (design defect) or the deployer (operational failure) depending on the failure mode.
Mandatory kill-switch requirements are moving from guidance to legally enforceable obligations across multiple jurisdictions.
Risk Signals
Emergency shutdown gates that have never been tested in production.
No documented procedure for who triggers a shutdown and under what conditions.
Shutdown gates that require network connectivity to function — they may fail when most needed.
Action Items
Test emergency shutdown gates in production at least quarterly under simulated conditions.
Document the shutdown trigger procedure and assign a named responsible person.
Design shutdown gates to fail safe — default to halt, not to continue.