Governance / 8 min read
The governance checklist before a model goes live
Nine questions that should have written answers before anything reaches a customer. Most of them have nothing to do with the model.
Who owns the output
Name a single accountable person for the system's behaviour in production — not a committee. If a customer receives a wrong answer at two in the morning, one named person should be the one who decides whether to disable the feature.
Ownership that is shared across a steering group is ownership nobody exercises under pressure.
Data: lawful basis, residency and retention
Establish the lawful basis for every category of personal data the system touches, where that data physically rests, and how long it is kept. For UK and EU subjects this is not optional, and for Indian data principals the Digital Personal Data Protection Act 2023 sets its own obligations.
State plainly whether any provider may use the data for training, and switch it off where it is possible to do so. Put the answer in writing in the client's own file, not only in the vendor's terms.
Evaluation, and what it is measured against
An evaluation set drawn from real cases, including the awkward ones. A baseline that represents current performance — usually human. A threshold agreed before results are seen, because thresholds agreed afterwards are negotiated, not set.
Regression coverage matters more than headline accuracy. The real risk is not that the system is imperfect at launch; it is that a routine change makes it quietly worse three months later.
Human review, logging and rollback
Define which outputs require human approval, and make the reviewer's workload realistic — a reviewer facing hundreds of items an hour is a rubber stamp with a job title.
Log inputs, outputs and the model version behind every decision, subject to your retention policy. Then rehearse the rollback: how the feature is disabled, who can do it, and how long it takes. An untested rollback is a plan, not a control.
Cost ceilings and incident response
Set a hard spend ceiling with an alert well below it. Usage-based inference costs fail in one direction only, and they fail fastest when something is looping.
Finally, write the incident path: who is told, in what order, within what window, and what the customer is told. Draft it while nothing is wrong. Nobody writes a good disclosure at speed.
If this is the kind of thinking you want applied to your own operation, a first conversation costs nothing.
Start a conversation