YOLO is not a security model
2026-04-29 · #ai #craft #systems
If your AI safety story is “the prompt says not to”, you do not have an AI safety story. You have a strongly worded suggestion aimed at a machine that is very good at continuing text.
That sounds flippant, but it is the whole problem. Models are getting better. The harnesses around them are getting better. AI labs are making models better at following instructions, and that work is real, useful, and worth doing. None of that turns a sentence in a system prompt into a permission boundary.
The prompt is not the perimeter
Rules in a prompt are instructions. They are not locks. They depend on the model interpreting them correctly, preserving them through a long context, resolving conflicts the way you intended, and not being nudged around by the next tool result, error message, or badly-shaped user request.
That is a lot to ask of sophisticated autocomplete.
The phrase is crude, and a bit unfair, and still useful. A model predicts and acts inside the shape you give it. It does not become a small responsible employee because one paragraph says “never run destructive commands without explicit consent.” That paragraph might help. It should exist. It should not be the thing standing between your agent and DROP DATABASE.
PocketOS is the obvious cautionary tale here, because every era of software gets the incident it deserves. Give an agent broad production access, tell it to be careful, then act surprised when carefulness turns out not to be an access control system.
Permissions are the real guardrails
Safety has to be boring and external. The model can be clever; the boundary should not be.
Three controls matter more than another paragraph of policy text:
- Scoped credentials - the agent gets the least access that still lets it do useful work. Read-only by default. Write access where needed. Production write access only when you can defend it without sounding like you are negotiating with fate.
- Human execution for irreversible actions - the agent can write the script, suggest the command, and even try to run it. The system should refuse. Deleting data, changing infrastructure, rotating secrets, spending money, sending emails, and deploying to production should require a human hand on the final step.
- Recoverable environments - sandboxes, staging systems, dry runs, backups, migrations with rollbacks, and audit logs. Not because containers are holy, but because reality is easier to survive when mistakes have blast radiuses.
This is not anti-agent. It is what lets agents be useful. A powerless agent is a chatbot with a clipboard. An all-powerful agent is an intern with root on production and a surprising amount of confidence. The useful version sits between those two unhappy objects: cruise control with your hands still on the wheel. The system can do the steady work, keep pace, and reduce the load. It does not get to decide when to leave the road.
YOLO has a bill
YOLO mode is not inherently reckless. It depends what the agent can touch.
If YOLO means “edit this local branch, run the tests, and make a commit”, fine. The worst case is usually a messy diff and a slightly longer afternoon. If YOLO means “operate directly against production with broad credentials”, the phrase becomes more literal than intended. You only live once. So does the database, if your backup story is mostly vibes.
The point is not to wrap every model in five containers, three approval queues, and a ritual sacrifice to compliance. That is how useful tools become theatre. The point is to decide which failures you are willing to absorb, then make the system physically incapable of causing the failures you are not.
There is a simple test I like: if the agent ignored every instruction in the prompt and used every permission it had, what could it destroy?
If the answer is “a scratch branch”, carry on.
If the answer is “customer data”, the problem is not the prompt.
Safe use is a systems property
AI is safe when it is used in a safe system. That sounds obvious, which is usually a sign we are about to ignore it.
We already know how to do this. We separate environments. We scope credentials. We review risky changes. We keep backups. We make dangerous operations explicit. We log what happened so the next incident is shorter and less mysterious than the last one.
The model does not remove those habits. It makes them more important, because the model moves faster than the human holding the keyboard used to. Speed is useful. Speed aimed at the wrong permission set is just a faster way to find out how good your restore process is. I hope it is better than PocketOS’s, where the backups were in the same blast radius as the database.
Use YOLO mode where the blast radius is honest. Let agents move quickly where mistakes are cheap, visible, and reversible. Put hard external boundaries around anything expensive, private, or difficult to rebuild.
Prompts guide behaviour. Permissions decide consequences.