Why did the AI agent stop responding?

The model version it depended on was retired by the provider. The application code was healthy; the dependency underneath it was gone.

Migrating to the current model, removing parameters the new version rejected, and adding output checks so the agent cannot silently degrade again.

What is the lesson for production AI systems?

Treat models and prompts as dependencies with lifecycles. Pin them, monitor them, and rehearse the migration before the provider forces it.

Our sales funnel is an AI agent. In June, it silently died.

The agent that runs our inbound funnel interviews prospects, writes the Discovery Brief, checks the calendar, and books the Workflow Audit. No human in the loop. We tell clients that production AI is an operations discipline. This summer, our own operation made the point for us.

The system

The agent is a deliberately boring stack: a Next.js app on Vercel, Claude behind the conversation, Postgres for leads and sessions, a small Cloud Run service holding the calendar integration, and Redis rate limiting in front of every endpoint that costs money. It qualifies, briefs, and books: the same job a human SDR would do, minus the follow-up emails nobody enjoys sending.

The failure

One day the chat simply stopped answering. No crash page, no alert: the stream opened and closed empty, which to a visitor looks like an agent with nothing to say.

The cause was not a bug in anything we wrote. The model our code requested had been retired by the provider. Model versions have end-of-life dates like any other dependency. Ours had passed while the system was quietly running, and every LLM call in the product began returning "model not found."

A model ID in your codebase is a pinned dependency with an expiry date. Almost nobody diaries it.

The two subtler failures behind it

Swapping in the successor model was a one-line change, and it produced two failures more instructive than the outage itself.

First, the API contract had tightened. The new model rejected non-default sampling parameters that the old one accepted. Three of our call sites set temperature values tuned years earlier; every one of them now returned a 400. The migration was not "change the model string". It was an audit of every parameter we had ever set.

Second, the behavior shifted. The new model was more capable and more thorough, and thoroughness is not always what you tuned for. Ours began helpfully writing out an internal summary, complete with internal recommendation taxonomy, directly into the client-facing chat. The old model had stopped where the prompt implied; the new one completed the thought. The fix was an explicit boundary in the system prompt. The lesson: a model swap is a behavior change, not a refactor.

The checklist we apply now

–Treat model IDs as dependencies with EOL dates. One constants file, never scattered, and the provider's deprecation calendar goes in the team diary the day a model ships to production.
–After any swap, audit every sampling parameter. Temperature, top-p, penalties: assume the new model's contract is stricter until proven otherwise.
–Re-run a golden conversation and read it. Errors are the easy failures. Behavior drift (tone, verbosity, boundary-keeping) only shows up when a human reads a full transcript.
–Alarm on empty streams, not just error codes. A stream that opens and closes with zero tokens is an outage that most monitoring setups call a 200.
–Rate-limit everything that bills per token. When a failure mode and a retry loop meet an LLM endpoint, the invoice is the alert you get last.

The point

"AI implementation" sounds like integration work: wire the model in, ship, done. What our June looked like is the truth: models retire, contracts tighten, behavior drifts, and somebody has to be carrying the pager when it happens. That somebody being us, for our own systems and for every system we hand over with evals and runbooks, is most of what forward-deployed means.