Plan first, test always
2026-04-23 · #ai #craft #testing
AI can write a week of code in an afternoon. That is the good news and the bad news in one sentence.
The good news is obvious. The bad news is subtler: a week of code you didn’t write, didn’t plan, and didn’t test is not a week of progress - it’s a week of unexamined assumptions compiled into a binary. It runs. It might even pass the happy path. And then, three Tuesdays from now, something you never considered will land on it and you will find yourself reading a stack trace through a function you technically authored but have never actually read.
The fix isn’t to use less AI. The fix is to bring back two disciplines we were already slipping on before the models showed up: planning and testing.
Planning is the part the model can’t do for you
A model can generate code that fits a prompt. It cannot decide whether the prompt was the right prompt. That part is still yours.
This is why “just start coding and let the agent figure it out” is such a seductive failure mode. The agent will figure something out. It is very good at figuring something out. But “something” is not the same as “the thing that makes sense given the three other systems this has to live next to, the migration we did last quarter, and the roadmap item nobody wrote down.”
Before you hand the keyboard over, write down:
- What you’re actually trying to do - in one sentence, in plain English, with no jargon.
- What already exists that it has to fit into - the shape of the surrounding code, the constraints you can’t violate.
- What you are explicitly not doing - the features you’re not adding, the edge cases you’re consciously deferring, the cleanup you’re not bundling in.
That last one is the one people skip. It’s also the one that keeps a small task from turning into a quietly sprawling one. An agent with no scope will happily expand until it hits the context window.
A rough heuristic I’ve come to trust: spend longer in plan mode than in edit mode. If the balance is the other way around, the plan wasn’t really a plan - it was a vague intention dressed up as one, and the editing is where you’re actually figuring out what you meant. That’s a very expensive place to do your thinking. Plan mode is also qualitatively different from edit mode - most harnesses prompt the model to explore the surrounding code, ask clarifying questions, and surface assumptions before anything gets written. Skip it and the model defaults to whichever premise felt obvious first, which is rarely the right one.
Testing is the part you no longer get to skip
There used to be two human excuses for under-testing code. The first was the honest-economics one: writing the feature already took hours, and writing tests for it would take more. The second was less honest but surprisingly common - the developer who believed their time was for writing code, not for running it, and who quietly trusted QA to find anything that mattered. Typing was the craft; exercising the thing they’d just built was someone else’s job.
There’s a sharper version of that second excuse worth naming out loud. Now that the model writes the code, a developer’s remaining contribution is mostly the prompt. But QA can prompt too. And QA has always been better than “I just write code” developers at the one part that still genuinely requires a human - running the thing, breaking it on purpose, noticing what isn’t quite right. A developer who outsources the writing to a model and the running to someone else is, pretty quickly, a developer who has outsourced themselves.
Both excuses are gone. If a model wrote the feature in twenty minutes, you didn’t spend those minutes typing - you have them back, and “I didn’t have time to test” just means “I didn’t choose to.” And if the model did the writing, running the thing isn’t a detour from the real work; it is the real work. Testing, in the age of generated code, is less about writing more asserts than about actually exercising the thing - running it, pushing its edges, pointing it at realistic data. That’s how you discover that the helper you glanced at actually swallows an error, or that the retry loop has no upper bound, or that the “obvious” type is subtly wrong at the edges.
A few rules I try to hold myself to:
- The model writes the code and the unit tests; I run it against the real world. Unit tests verify that the code agrees with itself - they catch the obvious regressions and the off-by-ones, and the model is pretty good at generating them. They don’t tell you whether the thing works when it’s plugged into a real database, a real upstream service, a real user path, or a payload shaped like production traffic. That part is still yours, and it is the part that actually matters.
- Integration and limits testing is where the real bugs live. What happens when the upstream is slow? When the input is twice as big as the agent was imagining? When two requests arrive at once? None of that lives in the unit suite. It lives in you, sitting in front of the running system, pushing on it.
- Read the model’s tests like a code review, not a receipt. Green checkmarks are not proof. I want to see the assertion, and I want it to be the assertion I would have written - and if it isn’t, I want to know why the model thought it was.
The loop that works
The workflow that has survived contact with reality, for me, looks roughly like this:
think: what context is missing from just the code
plan mode: prompt -> refine -> perfect
edit mode: implement -> run -> ship
(then loop)
Nothing clever. Start by thinking - not typing. What’s in your head about this change that isn’t in the code? The constraint from last week’s incident. The migration three quarters ago that still shapes the schema. The thing product asked for that nobody wrote down. Get that into the prompt before the model starts exploring - it can read the repo, it can’t read the Slack threads you read yesterday.
Then plan mode: prompt, let the model explore, answer the clarifying questions it comes back with, and keep refining until the plan is one you could defend with a straight face. Only then flip to edit mode and let it implement. Run the thing yourself - really run it, not just read the diff and trust the green checkmarks. Ship the small piece. Back to the top for the next one.
The temptation is always to skip straight to edit mode. A vague prompt, an agent implementing, a quick glance at the diff, ship. It works - right up until it doesn’t, and the failure, when it comes, is in a part of the system you never looked at because a machine wrote it and the machine seemed confident.
Confidence is not correctness
This is the thing I keep coming back to. Models are, by design, fluent. Fluency reads like competence. Competence reads like correctness. None of those implications are actually valid, but our brains accept them anyway, because for our entire lives “sounds like someone who knows what they’re talking about” has been a reasonable proxy for “knows what they’re talking about.”
It isn’t, anymore. The proxy is broken. The only things left that still correlate with correctness are the boring ones: a plan you can defend, code you have read, and tests that run against something real.
Use the models. Use them a lot. Just don’t let the speed convince you that the old disciplines got cheaper. They got more important - because the cost of skipping them is now measured in how much unreviewed code you’re sitting on, and that number grows faster than it used to.
Plan the thing. Run the thing. Then ship it.