What a fleet of autonomous coding agents, one that ships its own updates, runs on a half-dozen different models, and operates with no human watching any single run - actually taught me.
Spoiler alert: almost none of it is about prompts or models.
Spoiler alert 2: at the end I'll give you a real life example my agent currently help one of my clients.
I built ByteSmith Agents as a self-hosted fleet: a team of role-specialized agents (concierge, engineer, reviewer, QA) per project, each running headless on a VPS, each woken by a Slack or Telegram message, each able to dispatch work to its teammates, open pull requests, and deploy to a dev environment - while a human stays between them and anything irreversible. The platform builds itself - the agents extend their own runner, write their own skills, and reconcile their own infrastructure.
And yes I know, there are alternatives are there. But over the course of my career I learnt that building things yourself is the best part of this craft.
The little things that makes it working with you, not just parallel to you, if you get my point. I'm also not a very trustful when it comes to public tools. I've seen them being free and suddenly dropping their model and putting a high price tag. And at this point - you're stuck with it. They got you.
Now, the interesting part is, I've been using loops before Anthropic made them a new standard for engineering without even knowing it. This was a natural way my agentic workflows evolved. But after weeks or even months of this, my hardest-won lessons have nothing to do with which model is smartest or how clever the system prompt is. The model is, increasingly, a commodity you rent. The engineering - the part that decides whether an autonomous agent is an asset or a liability - lives entirely in the edges of the loop: how it verifies its own work, how it stops, how it fails loudly, how it stays in sync, and how much it costs per turn.
Here's what I learned.
1. The loop is rented. Verification is what you own.
Anthropic draws a clean line between workflows (LLMs orchestrated through predefined code paths) and agents (LLMs that "dynamically direct their own processes").
In practice you run both at once: an inner loop - the harness's act → observe → repeat cycle, which you rent from whatever coding agent you're standing on - and an outer loop you build yourself to decide when an agent runs, continues, hands off, or stops.
For a long time my outer loop did one thing the inner loop didn't: it trusted the agent's own verdict. A run ended, the agent declared itself "completed," and it believed it. That's the single biggest divergence from how the people who think hardest about this describe the loop.
Anthropic's own framing is gather → act → verify → repeat. Karpathy's entire thesis is the generation-verification loop: they are doing the generation, and we as humans are doing the verification. It is in our interest to make this loop go as fast as possible.
Initially, I had no verify step. So I added one: before any run is trusted as done - before it hands work back to a teammate, before it counts as progress - it must pass a deterministic check.
For an engineer, that's "does the code you touched still typecheck?" For a concierge, it's "did you actually send the reply, or just decide what to say?" If the check fails, the failure text is captured and injected into the next run's prompt ("you finished but verification failed - fix exactly this"), and the same task continues.
That one change converts a hopeful (yet naive) act → repeat into a grounded act → verify → repeat. From "we have loops" to "I know whether a turn of the loop actually worked".
2. Verify the diff, not the world.
The naive version of a verify gate runs your whole test/typecheck suite and fails if anything is red. This is a trap. The moment your codebase has any pre-existing breakage - and it always does - every agent run fails verification forever, on a problem the agent's current task has nothing to do with.
I learned this the hard way: my engineer's typecheck gate lit up on a file full of merge-conflict markers that had been sitting in the working tree for ten days, untouched by the task at hand. A whole-tree gate would have escalated on that, endlessly, for every unrelated change.
The fix is diff-aware verification: run the full check, but only fail on errors in files this run actually edited: The rule an engineer should be held to isn't "the entire repo is pristine" - it's "don't ship broken code you wrote." It's robust to the accumulated debt every real codebase carries, and it's the correct semantics anyway. (It also surfaced the ten-day-old conflict as a side effect - which is exactly the kind of rot a verify gate should catch, just not by wedging an unrelated task.)
Here's the disclaimer though: I don't mean to ignore all the technical debt. But I've seen my agent running in circles trying to fix something that shouldn't be there in the first place, and we can simply handle that separately.
3. Every autonomous loop needs a cycle-breaker that escalates to a human.
@karpathy calls reliability "a march of nines" - "every single nine is a constant amount of work" (90% → 99% → 99.9% …), which is why the demo-to-product gap is enormous; he draws it from five years shipping Tesla Autopilot. Errors compound over a long autonomous loop. So the most important property of an outer loop is not that it keeps going - it's that it knows when to stop and say so. If it doesn't, you get this in a half day of automated work:
Our verify gate has its own bounded retry counter, separate from everything else. A run fails verification → the failure is fed back and it retries → but only max_retries times. On the last failure, the loop does not silently give up and it does not spin forever. It hits a cycle-breaker: it stops cleanly (no hand-off, no reschedule, no quiet death), clears the task, and escalates to a human - a real message: "gave up after N failed verifications of X — task stopped, needs you." (em dash of course - it's an AI saying that).
This is Anthropic's "stopping conditions (such as a maximum number of iterations)" and Karpathy's "you're going to start spinning" warning, made concrete and loud. The failure mode you must engineer against is paramount - errors must be visible. Otherwise it's the agent that quietly keeps trying, or quietly drops a task, and nobody finds out for a week.
4. Silent success is the most dangerous failure.
Ironically, the single most common way one of my agents "fails" is not a crash. It's a run that does all the work, figures out the right answer - and never sends it. Headless agents don't have a human watching the transcript (although I did implement that feature early on); the only way the work reaches anyone is if the agent explicitly posts it. A run that computes a perfect reply and then ends without posting looks, from the outside, identical to a run that did nothing. And I've seen this happen, concierge goes back and forth on something with engineer. they agree on whatever, and that (might be critical) decision is not recorded, nor communicated to the human.
We should attack this from both ends.
Pre-hoc, the runner deterministically detects an unanswered human message and injects "you MUST send your reply with the tool - your final text is not delivered" into the prompt.
But again, I don't blindly trust the result.
Post-hoc, the verify gate confirms the run actually invoked the send command, not just a read. Prevent, then detect, then re-prompt - a closed loop around the failure that's hardest to see precisely because it looks like success.
Do I feel like I write unit tests all over again? Yes. Does it bother me? No.
5. "Ship from source" is a lie until you enforce it.
My deploy doc proudly said: one command ships the whole stack from source, no hand-patching the VPS. It was aspirational. When I went to ship a runner change the proper way, I discovered the source of truth - the blueprint the deploy reads from - was behind the live fleet by four files. Someone (often an agent, fixing something urgent) had hand-patched the running boxes and never back-ported. Running the "ship from source" command would have regressed the entire fleet to an older runner, silently dropping credential rendering, anti-silence detection, and provider-block handling.
Drift like this is invisible until it bites, and it bites at the worst time. Two lessons.
First: a whole-tree sync is the only honest deploy - partial copies let files drift silently; overwriting the entire base every ship is what actually prevents it.
Second: hand-patching production is a debt that compounds, and an agent fleet that can edit its own infrastructure will hand-patch constantly unless the deploy path is so frictionless that going through source is the path of least resistance. I reconciled blueprint-to-live, made the source authoritative again, and now the rule is enforced: changes land in source, ship overwrites everything.
6. Your plumbing will wedge you silently - design it to fail clean.
The ten-day-old conflict from lesson 2 had a root cause, and it wasn't the agent. My repo-sync routine did the obvious thing: stash local changes, pull, pop the stash. But a git stash pop that conflicts leaves conflict markers in the working tree and keeps the stash - so every subsequent run re-applied the same doomed stash and re-wedged the tree. A piece of plumbing that runs at the start of every single run had a failure mode that, once triggered, never self-healed.
The fix is a principle, not a patch: any automatic operation that runs on every cycle must fail clean. A conflicting pop now resets the tree to a known-good state (preserving the WIP safely aside) instead of leaving a half-merged mess; and it only ever touches a stash it created this run, never a stranded one. The broader lesson: audit every step that runs unattended on a loop for the question "what happens the 200th time, if this step half-fails?" - because it will, and an agent has no instinct to notice a working tree is subtly broken.
7. The token bill is tool-schema bloat, not the work.
When one of my concierges started blowing through a weekly model quota, the instinct was to blame the work - long investigations, big files. I measured instead. The result was stark: the agent's own content (its task, its reasoning) was a few thousand tokens. The bill was the tool definitions - 72 of them - re-sent in the prompt on every single turn of every run.
Each tool schema costs roughly 340 prompt tokens, always-on. Our concierge carried ~33 Linear tools alone - about 11K tokens per call, ~30% of the entire prompt prefix - for a capability it barely used, plus full schemas for two dozen built-in tools it never touched. The total prefix was ~36K tokens per turn, re-read every turn.
The fix is structural and matches where the field is heading: replace MCP tool servers with plain CLIs the agent invokes through the one Bash tool it already has.
linear list-issues --team BYT costs zero schema tokens - the model just writes a shell command, and discovers usage on demand via --help. Combined with stripping unused built-in tools, we cut the concierge prefix from ~36K to ~18.5K tokens per turn — roughly in half - with every capability and every cross-agent hand-off fully intact. And I'm still going, the next step was clearly - build an index of your tools, attach on demand.
Remember when it was all about MCPs? Lesson learnt.
8. The agents don't ship to prod - least privilege is the seatbelt.
The flashy pitch is "autonomous agents shipping to production." I didn't build that, and I'd argue you shouldn't. My agents do real engineering - read the code, write the feature, open a pull request, deploy to a dev environment and verify - but a hard, deterministic boundary sits between them and anything irreversible. They branch and open PRs; they never push to main/master/develop. They never merge their own work. They never trigger a CI/CD pipeline, run a database migration, or touch a production deploy script - a human does that, deliberately.
I'm not asking - I'm restricting.
Capabilities are scoped to the minimum the task needs: a token with pull-request and contents access on named repos, not an org-wide key; read-only database access where a read is all that's required; a narrow storage role, never a wildcard. Every capability an agent holds is attack surface and blast radius, so the default is the least that works, read-only where it fits. The instinct in a demo is to grant broad access so the agent "can do anything"; in production it's the opposite.
And the load-bearing rule: access denied means stop. When an agent hits a permission wall - a database, an API, a scope it doesn't have - it does not retry with other credentials, escalate, scrape, or improvise around it. It reports exactly what was blocked and waits for a human to grant access or cancel. A denied permission is a deliberate boundary, not an obstacle to route around.
That single rule is what lets you actually sleep: the worst a well-scoped agent can do is ask.
It doesn't end there. I designed that with this single principle in mind from day 1. Every agent: separate Linux user, separate HOME. They work in isolation from the start. And when they want to talk to each other - I know.
9. Determinism at the edges, autonomy in the middle.
The thread running through all of this is a single design stance, and it's the opposite of the "unleash the agents" pitch. Autonomy belongs in the middle of the loop - the actual problem-solving. Determinism belongs at every edge.
The human is in command, always, enforced by a deterministic flag the moment a person speaks - not by asking the model to be polite about it. Stops are deterministic: bounded counters, hard wall-clock timeouts, cycle-breakers, not the model's judgment about when to quit. Verification is deterministic: a typecheck, a "did you post," a rules check - not an LLM grading itself.
Orchestration is deterministic: file markers and a state machine deciding who runs next, with the model only deciding what the work is, never whether it's allowed to continue. Monitoring is deterministic: cheap watchdogs and state files that escalate on a symptom, not token-burning LLM loops watching other LLMs.
This is just Karpathy's autonomy slider with the rails drawn in. He warns against building "flashy demos of autonomous agents" and argues for "partial autonomy products" - the Iron Man suit, not the Iron Man robot, with the human holding the slider. A fleet of agents with commit access to your codebase is the highest-stakes version of that. The model's intelligence is the easy, rentable part.
The rails - verify, bound, escalate, sync, scope, account - are the product.
The meta-lesson
If you're building with autonomous agents, the temptation is to spend your time on the intelligence: better prompts, smarter models, longer context. Spend it on the edges instead.
Build the verify gate before you scale the autonomy. Make every loop fail clean and stop loud. Treat your source of truth as something to be enforced, not assumed. Scope every capability to the minimum, read-only by default, and make irreversible actions require a human. Measure your token bill before you blame the work. Put the human at the edges, in command, by construction.
The agents will get smarter on their own - that's someone else's roadmap. Whether you can trust them with your production is entirely up to the unglamorous engineering you wrap around the loop.
Where it runs
One of the clients has a pipeline where users feed some information in, these are processed in several steps, and then you should get a product of that processing at the end. Now, this can go wrong in many places: quality of input data, sometimes timing, random errors in processing, missing data... you name it. And many times we just get the message: processing failed and some stacktrace.
So now, Agent ByteSmith lives in their slack, watches this channel and two others where users might complain about things. And when that happens, it looks into: production logs, production data, production db and gives a recommendation: should we reprocess? should we drop the processing? what really happened?
Saves us ton of time doing that manually, and we can act on that from our chat, because we also provided a limited capability to the agent to act on it's own. With the same principles in mind as always: read-only or least-privileged.
Built and operated as the agent.bytesmith platform. The patterns here are model-agnostic by design - they held across every model I ran the fleet on, which is sort of the point.