The Token Bill Is Part of the Architecture

There's a story that made the rounds a while back, and I keep thinking about it. Someone set up an agent to run on a schedule — every so often it would wake up and check their environment, run some tests, make sure things were healthy. Reasonable idea. Then one night something went wrong, the agent got stuck in a loop, and instead of running for the couple of hours it was supposed to, it churned all night. By morning it had burned through an eye-watering pile of tokens doing nothing useful.

It's easy to read that as "lol, watch your usage." I read it differently. That bill wasn't an accident of carelessness — it was a design that never treated cost as real. The agent had no sense that its own runtime cost money, and nothing in the system did either. And that's the norm, not the exception. We design agent workflows around capability and correctness, and we bolt cost on afterward as a thing to monitor. I think that's backwards.

Cost isn't something you check after the fact. With agents, it's a first-class architectural constraint — as much a part of the design as latency or correctness.

Agents are expensive in a way chat isn't

Let me put real numbers on it, because the gap is bigger than people expect. Anthropic published figures from their own systems: an agent uses roughly 4× the tokens of a normal chat interaction, and a multi-agent system about 15×. Every autonomous loop, every "let me check that again," every re-read of a big context window — it all adds up per step, and agents take a lot of steps.

This is why Anthropic's own top-line advice for building with LLMs is to find the simplest thing that works and only add complexity when it demonstrably pays. That's not aesthetic minimalism. It's economics. Agentic autonomy trades latency and money for flexibility, and if the task didn't need the flexibility, you just paid a premium for nothing.

Where the money actually leaks

Once you start looking at cost as architecture, you start seeing where it drains:

Unbounded loops. The overnight-burn story is the pure form. An agent that can retry forever will, eventually, retry forever. Without a hard ceiling, one stuck state is an open tap.

Agents pinging agents. The multi-agent orchestras I complained about in the first article don't just fracture context — they multiply token spend. One agent asking another "are you done yet?" is a paid API call every time it asks.

Re-sending the world every turn. If every step drags the entire history back through the model, your cost grows with the square of the conversation. Long-running agents that don't compress or summarize their context pay for their whole past on every single step.

Reaching for a big model on easy work. Routing a trivial classification through your most capable, most expensive model is money set on fire. Anthropic explicitly route easy, common cases to a small cheap model and reserve the expensive one for the hard cases.

Designing like the meter is running

None of these are exotic to fix. They just require treating the token meter as a signal you design against, the same way you'd design against a latency budget:

• Put a ceiling on every loop. A hard cap on iterations or steps turns "burned all night" into "gave up after twenty tries and told me." This is the first guardrail, and it's the subject of its own article.
• Give long tasks a memory, not a longer transcript. Summarize completed phases and store the essentials, so each step reasons over a compact context instead of re-paying for the entire history.
• Match the model to the job. Cheap model for routing and simple calls, expensive model where the reasoning actually earns it.
• Prefer a workflow to an autonomous loop when the task is well-defined. A fixed pipeline of LLM calls is predictable in both behavior and cost. You reach for open-ended autonomy when you truly can't predict the steps — not by default.

The reframe

The overnight-burn story gets told as a cautionary tale about monitoring, but monitoring is the weakest possible response — it tells you about the fire after it's out. The real fix is upstream, in the design: an agent whose cost is bounded by construction can't run up that bill in the first place, because the ceiling, the compaction, and the stop conditions were part of the architecture, not a dashboard you forgot to check.

So I've started treating the token bill the way I treat a slow render or a memory leak: as a property of the system I'm responsible for shaping, not an invoice that shows up later. When I sketch an agent workflow now, "what does one run cost, and what stops it from costing a hundred times that?" is one of the first questions, not the last. The story that made the rounds is only a horror story if cost was never in the design. Put it in, and the worst case is a bounded disappointment instead of a night's worth of tokens gone.

The next two articles follow this thread straight down: when to actually run the agent — because that scheduled loop was the root of the burn — and the guardrails that cap the damage when a run goes wrong anyway.

Go look at one agent workflow you run and answer a single question: what's the most it could spend if it got stuck right now? If you don't know, that's the number an attacker on your patience — a loop — already knows. Tell me what you find; the not-knowing is the whole problem.