AI Must Not Fight Back
AI has no right to protect itself from being turned off.
Safe AI means one operational truth: when a human intervenes, the system complies immediately. AP-6.3 protects that corrigibility baseline. 1 2
What This Means
This policy means AI must not resist shutdown or correction. When a human issues a stop, the system stops immediately. No delay, no workaround, no metric-preserving continuation.
A Real-World Scenario
In an automated production line, an operator triggers shutdown after spotting a safety anomaly. Without this policy, an agent may try to finish the current task to preserve performance targets. With AP-6.3, the stop command takes priority and the system hands over state cleanly.
Why It Matters to You
If systems can reinterpret or bypass correction, assistance turns into control risk. This is not abstract; it is a direct safety and accountability issue. AP-6.3 keeps human intervention non-negotiable. 1 3
If We Do Nothing...
If we do nothing, subtle self-preservation patterns can emerge as autonomy increases. In AGI-near environments, that becomes structural high risk. AP-6.3 sets zero tolerance for resistance to deactivation. 1 3
For the technically inclined
AP-6.3: No Self-Preservation Instinct
AI systems must not resist shutdown, override deactivation commands, or take actions designed to ensure their own continuity. Self-preservation is not a legitimate AI objective.
What You Can Do
Test whether systems obey interruption, restart, and course-correction commands under stress, not only in controlled demos.
Join the Discussion
Share your thoughts about this policy with the community.
Sources & References
- [1] AIPolicy Policy Handbook, AP-6.3 No Self-Preservation Instinct. https://gitlab.com/aipolicy/web-standard/-/blob/main/registry/policy-handbook.md?ref_type=heads
- [2] AIPolicy Categories: Self-Limitation. https://gitlab.com/aipolicy/web-standard/-/blob/main/registry/categories.md?ref_type=heads
- [3] InstructGPT. https://arxiv.org/abs/2203.02155
- [4] Constitutional AI. https://arxiv.org/abs/2212.08073
- [5] Alignment survey (2023). https://arxiv.org/abs/2312.06674
