The short answer

Production AI is a system real users depend on for real work, engineered to stay dependable. The definition is checkable: integrated with the systems of record, handling exceptions deliberately, proven by evals, observable when it degrades, access-controlled, and owned by someone who can operate and change it. Miss any of the six and what you have is a demo with traffic.

"In production" has become the most inflated phrase in AI sales, claimed equally by systems that process a company's invoices nightly and by prototypes that survived one supervised week. The phrase deserves a definition sharp enough to expose the difference.

The definition

A demo shows the model can do the task once, on chosen inputs. A pilot shows it can do the task for a while, under supervision, with the hard parts postponed. Production means the postponed parts exist and are load-bearing: the system runs on real data, inside real workflows, for users who did not build it and will not forgive it, with the machinery that keeps it true over time. The gap between those states is where the overwhelming majority of AI initiatives die, which is precisely why the word gets borrowed by systems that never crossed it.

The six properties

  • –Integrated. It reads from and writes to the systems of record: the calendar, the database, the queue. A system that needs a human to copy its output somewhere is a suggestion engine, not production.
  • –Exception-handling. The weird cases are designed for: routed, escalated, or refused on purpose. Production systems are defined by what they do with the inputs nobody predicted.
  • –Proven by evals. A fixed set of realistic cases, run on every change, with thresholds that gate release. Evals are the difference between "it works" as a feeling and as a number.
  • –Observable. Someone can tell it degraded before the users do, including the silent failures: the empty stream, the drifted tone, the model update that changed behavior overnight.
  • –Access-controlled. It knows who may ask it what, and what it must never do for anyone, and those boundaries survive hostile inputs, not just polite ones.
  • –Owned. Somebody can operate it, change it, and survive its dependencies retiring: runbooks, documented prompts, and the full handover if a vendor built it.

Production systems are defined by what they do with the inputs nobody predicted.

Why it is rare

None of the six properties demo well. Every one of them is invisible in the sales meeting and decisive in month three, which is why they get postponed, and why the postponement is usually permanent: crossing from pilot to production means building them all at once, against an audience that already saw the demo and assumes the work is done. The attrition that results is measured across six studies, and it is not close. We run our own funnel on a production agent and published the June incident that tested these properties on us: the failure was silent, the recovery ran through evals and monitoring, and the lesson was that production is an operations discipline, not a launch event.

What buyers should do with this

Use the six properties as a specification and a filter. As a specification: write them into any engagement's definition of done, so "production" is a checklist rather than a vibe. As a filter: when a vendor claims production AI experience, ask to use the system, then ask how it is monitored, what the evals check, and what broke last quarter. Operators answer with specifics and dates; the rest pivot to a demo. The full interrogation kit is in how to choose an AI development partner.