Someone Else's Release Notes, Your Production Incident: Governing Vendor Model Drift
Photo: U.S. Navy photo by Aaron Schoenfeld, Public domain, via Wikimedia Commons
The Release You Never Approved
On a Tuesday morning in the middle of a quarter, a vendor updates the model that powers the summarization panel inside your contract review tool. No maintenance window is announced to your team. No one on your side opened a change request. Yet the summaries your legal operations staff reads that afternoon are shaped differently: longer, more hedged, occasionally citing clauses in a different order. By Thursday, three reviewers have quietly started double-checking every output by hand, which is precisely the behavior the tool was purchased to eliminate.
This is the defining operational hazard of buying AI as a feature rather than building it. The vendor ships continuously. Your organization absorbs the consequences discontinuously, usually as anecdotes from frustrated users rather than as incidents with timestamps.
The argument here is straightforward: any vendor model change that alters the behavior of a business process should be governed as a production event — classified, tested, communicated, owned, and reversible. Not because vendors are careless, but because their release cadence and your accountability are on different clocks.
Why We Did Not Change Anything Is Not a Defense
When an automated decision misroutes a claim or a generated response reaches a customer with the wrong tone, the internal explanation that no one on the team deployed anything is true and useless. Regulators, auditors, and affected customers do not distinguish between code you wrote and behavior you licensed. The accountability lands where the process lives.
Three structural facts make this harder than conventional vendor management:
- Model updates are often silent by design. Many providers treat version promotion as an infrastructure concern rather than a product change, so the changelog entry may be a line item rather than an announcement.
- Behavior changes are statistical, not binary. A traditional upgrade either works or throws an error. A model update shifts the distribution of outputs, degrading a small but consequential slice of cases while the aggregate looks fine.
- Your prompts are part of the system. Even a pinned model can drift if the vendor adjusts a system prompt, a safety filter, or a retrieval configuration on their side of the boundary.
Four Controls That Belong in Both the Contract and the Runbook
Version pinning with a deprecation calendar. Insist on the ability to pin a specific model version per tenant or per endpoint, and require written notice — sixty days is a reasonable floor for business-critical workflows — before a pinned version is retired. A pin without a calendar is a countdown timer with no display.
A regression suite built on your own tasks. Generic benchmarks tell you almost nothing about your contract summaries or your claims triage. Assemble a golden set of two hundred to five hundred real, de-identified cases with agreed expected outcomes, and re-run it whenever a model version changes. Automate the scoring, but keep a human review step for the cases that matter most.
Notification commitments in writing. Ask specifically how model changes are communicated, at what granularity, and to whom. A status page is not a notification channel. Name an address on your side that receives change notices and route it to a monitored queue rather than an unread shared mailbox.
A named owner for AI behavior. Every AI capability in production needs a person whose job includes answering the question has this changed? That is a role definition, not a committee. Without it, the golden set goes stale, the changelog goes unread, and the first signal of drift is a complaint from an operations manager.
Classify Model Changes Like Releases
Not every update warrants a change advisory board. A useful three-tier scheme:
- Tier 3 — cosmetic or low-stakes. Internal drafting assistance with human review at every step. Log the change, re-run the golden set on a sample, no gate.
- Tier 2 — process-affecting. Anything that routes work, drafts customer-facing text, or scores a case before a human decision. Requires a regression run, a canary period with a limited user group, and a rollback path to the prior pinned version.
- Tier 1 — consequential. Outputs that feed regulatory reporting, credit or eligibility determinations, safety-adjacent recommendations, or automated communications. Requires pre-deployment evaluation against the golden set, a blackout window around critical business dates, an explicit sign-off by the business owner, and a documented fallback procedure that does not assume the vendor will restore the old version on request.
Map these tiers onto the change management process you already run for internal releases. The point is not to invent a parallel bureaucracy but to extend an existing one across a boundary it was never designed to cross.
The Question Procurement Rarely Asks
Most AI procurement evaluations concentrate on capability demonstrations and data handling terms. Far fewer ask the operational question: what does the vendor's model lifecycle look like over a three-year contract, and what rights do we have when it changes? Ask for the deprecation policy, the notice period, the versioning scheme, the evaluation methodology the vendor uses before promotion, and whether tenant-level pinning carries a premium.
Add one more: ask what happens to your data — prompts, outputs, feedback signals — when a model is retired. Continuity of the workflow depends on continuity of the artifacts around it, and that detail is almost never in the summary deck.
What Leadership Should Hear
AI features do not remove operational responsibility; they relocate it. The vendor operates the model, but the business process, the customer relationship, and the audit exposure remain yours. Governing model drift is therefore not a technical nicety. It is the price of adopting capability you do not control, and it is considerably cheaper than explaining, after the fact, why a workflow nobody on your team shipped started producing different answers on a Tuesday.