What AI Agents Are Actually Good For

I've been experimenting with AI agents on my own projects, and the part that interests me most isn't the code they type. It's that they can stay with a problem long enough to inspect a few repos, read the handoff, change a thing, run the tests, and tell me exactly what happened.

That is a lot more useful than autocomplete. It's also where you can get into trouble.

An agent can produce a very convincing “done” while the thing that matters is still untested. I ran into that distinction while getting Comfy Pass ready. There are five apps to connect, a central account, billing, and a separate existing subscription that must keep working. A passing unit test in one repository doesn't prove that a person can pay, receive access, use an app, and cancel without a surprise. The joins between those steps are the product.

So my view of the agent trend is pretty unglamorous: the better they get at doing work, the more precise I have to be about what “working” means. Which branch was tested? Was the code deployed? Did we check a real browser? Is a result from a mock being presented as a result from production? Those questions are less fun than asking an agent to build a feature. They're also the questions that stop a plausible demo from becoming somebody else's problem.

I like giving agents a bounded job and a place to leave evidence. Read the current state, make a change, explain it, run the relevant checks, and hand off the parts that still need a human or a real environment. When several projects touch the same customer path, I want the handoffs to say what they didn't verify just as clearly as what they did.

I don't want to pretend the answer is “never trust AI.” I use it because it lets me explore ideas and build things I otherwise wouldn't have time to try. It can help me get through the boring parts and spend more attention on the design and the actual result. That's exciting! I also don't want to pretend the agent owns the consequences of a billing mistake or a misleading page. I do.

The useful shift, to me, is from “AI wrote this code” to “I can ask for a piece of work and inspect the evidence.” That still takes judgment. Maybe more of it than before, just in a different place.

And yes, I am using agents to help with this site. If you find a weird sentence or a broken link, tell me. I will be the one fixing it.