Back to all posts

Test on Change, Not on a Timer

Aug 12, 2026
9 min read
Test on Change, Not on a Timer

Go back to the overnight-burn story for a second, because the detail that matters most is the one everyone skips past. The agent was on a schedule. It woke up on a timer to check the environment, over and over, whether or not anything had actually changed. The loop was the accident. The schedule was the design flaw that made the accident possible.

And it's such a natural design flaw. When you want an agent to keep something healthy, the first thing your brain reaches for is a cron job: "every N minutes, go check." It feels responsible — always watching, always on. But a timer is a fundamentally lazy trigger. It fires on the passage of time, which is almost never the thing you actually care about.

A timer runs when nothing happened

Think about what a scheduled agent actually does most of the time: it wakes up, looks at an environment that is exactly as it left it, does a bunch of work to confirm nothing changed, and goes back to sleep. You're paying full agent price — remember, 4× a chat and up — to repeatedly discover that nothing is different.

And on the rare occasion something is wrong, the timer is no help there either. If the environment breaks thirty seconds after a run, your schedule sits on that failure until the next tick. So you tighten the interval to catch problems faster, which means even more empty runs, which means more cost and more surface area for exactly the kind of loop that burned all night. The timer is bad at both jobs: it's wasteful when things are fine and slow when they aren't.

A clock is not an event. Waking an agent on a schedule means waking it for no reason, most of the time — and paying for the privilege.

Wake it on a reason

The alternative is almost obvious once you name it: run the agent when something changes. Not "every five minutes," but "when a commit lands," "when a deploy finishes," "when a check goes red," "when this file is updated." The trigger is an event that carries actual meaning, and the agent's run is a response to that meaning.

This flips every problem the timer had:

  • No empty runs. No change, no event, no run. You stop paying to confirm the status quo.
  • Immediate response. The agent reacts the moment the thing it cares about happens, not on the next tick.
  • Naturally bounded work. One event produces one run with a clear scope: "here's what changed, deal with it." That's a far easier task to reason about — and to cap — than an open-ended "go check everything."
  • Debuggable history. Every run traces back to a specific event. When something misbehaves, you know exactly what set it off, instead of squinting at a timeline of identical scheduled wake-ups.

Testing is the cleanest example. Running your whole suite on a timer is nonsense — nothing changed, so nothing new can break. Running it when code changes is the entire point. The event is the reason the test is worth running. That principle generalizes: the right moment to act is the moment the world became different, because that's the only moment new information exists.

The honest exceptions

I don't want to pretend timers never have a place. Some work genuinely is periodic: a nightly summary, a daily digest, a report that's tied to the calendar rather than to a change. If the business meaning is "once a day," then a schedule is correct — the time itself is the event.

But even then, two rules hold. First, it should still be bounded hard, because "runs on a schedule" and "can run forever once started" are independent properties, and the second one is what bites you. Second, be honest about whether the task is actually periodic or whether you reached for a timer because it was the easy primitive. Most "check the environment every N minutes" jobs are change-driven work wearing a schedule's clothing.

The reframe

The shift from timer to event is small in code — a webhook instead of a cron entry — and large in consequences. It's the difference between an agent that runs because a clock ticked and an agent that runs because something happened that it should respond to. The first is a machine idling in your driveway, burning fuel to go nowhere. The second only starts when there's somewhere to go.

That overnight bill wasn't really caused by a loop. The loop was the spark. The schedule was the pile of dry wood that let a single spark burn until morning. Trigger the agent on change instead, and most of that wood is never there to catch — the agent simply isn't running when there's nothing to do. What you still need is a firebreak for the runs that do go wrong, and that's the last piece: the guardrails that cap the damage.

Find one scheduled agent job you're running and ask what event it's really waiting for. If you can name the change it's checking for, you can trigger on that change directly — and delete the clock. Tell me what the event turned out to be; naming it is usually the moment the cron job stops making sense.

Telegram

More than a blog post

I share frontend news and the reasoning behind it throughout the day. Pick the language that feels natural to you.

Need to discuss your project? Get in touch.