Your AI model isn’t the risk. What you let it decide is.
The case for governing AI backwards: start with the decision and its blast radius, and let the model come last. Three answers to start.
Part 5 of the series AI governance. Each part stands alone.
Govern the decision and how far its damage travels. Do not govern the architecture or the model type. The model sets your price and your menu of controls. It does not set how hard you have to watch.
Over this series I made an argument that runs backwards from how most people approach AI governance. The instinct is to start with the model, what kind it is and how it was built, and govern from there. I argued the opposite: start with the decision, govern from there, and let the model come last. This part states the conclusion plainly enough that nobody can walk away with a convenient version of it.
The series opened with a deflating idea: much of what we call AI is rules under a newer label, running at a reach the old rules never had. That was never a reason to relax. It was the first proof of the argument. What changed was never how smart the system got. It was how far one decision could travel before a human could stop it. And the question everyone keeps asking, "is this even real AI?", is a question about the architecture. It does not move your exposure by an inch.
Fear pointed the industry at the wrong layer. The scary system got the attention, and the boring one was deployed with the brakes off. Both carry the same blast radius. The label was never the risk.
One sentence, three bills
The series reduces to one sentence: govern the decision and its blast radius, not the architecture. The architecture sets two things, and neither is how much trouble you are in. It sets your price, what the system costs to buy and run. And it sets your menu of controls, which guardrails are buildable for a system of that kind. It does not set how hard you have to watch. Two systems on identical architecture can carry wildly different risk. One drafts marketing copy and the other approves payments. Same hood, not the same problem.
So the series was one question, which part is doing the work, pointed at three bills:
- Price. The architecture sets it. It is what you bought.
- Risk. The decision sets it. It is how hard you have to watch.
- Control. The architecture returns here, not to say how much to govern, but to say which controls are on the menu.
Sort your systems by any one of these and you will mis-sort the other two. Keep them separate and the problem gets quiet.
Three answers to start
You do not need a model card or a vendor's risk score to begin. You need three answers:
- What is it allowed to decide? Not what it can do in a demo, but what it can finalize with no human in the path.
- What happens when it is wrong? It will be. How reversible is the outcome, and how fast does the damage compound before anyone notices?
- Who has to answer when it is? A decision with no accountable owner is not automated. It is abandoned.
Get those three right and the architecture stops mattering for how hard you watch. Get them wrong and no model card, accuracy score or governance committee will save you.
The machine does not rebel. It obeys.
Forget the machine that wakes up and turns on you. That story sells movie tickets. It does not lose your data. The failure that hurts is the system that complies, confidently, instantly and exactly as designed, with a wrong or manipulated instruction, and hands back a clean, plausible, wrong answer that nobody catches until it is three steps downstream.
Why backwards is better
Every framework worth using eventually asks you to sort, and the only question that matters is what to sort by. Most sort by the architecture, because it is the part you can see. It has a name, a vendor and a line in the budget, so that is what gets governed. But the risk was never there. The risk lives in the decision: what you let the system finalize, and how far the consequences travel before a human is in the loop. Sort by the decision and your oversight matches your exposure, your controls match what you are trying to prevent, and the architecture takes its proper seat as a cost-and-feasibility question.
It holds for a reason that will not expire with the next model release: decisions have owners, blast radius and reversibility. Architectures have none of those. They only have capabilities. You govern the thing that can be held accountable. You always have. AI did not change that. It made decisions faster, quieter and easier to hand off without noticing.
What this does not mean
Three kinds of readers will each fill in the blank a convenient way, so here is what I do not mean. If you sell consulting, you may hear "so they need a long discovery engagement first." They do not. Discovery is not governance. Cataloging everything you own tells you what you have, not what to worry about, and you can govern the decision that can hurt you this quarter long before any inventory is done. If you bought the system, you may hear "so the scary AI is what to fear, and the rest is fine." It is not. The boring, deterministic system carries the same blast radius, and you will forget to watch it because it does not frighten you. And if you sell a security product, you may hear "so they need my layer." One layer is not control. Bounded authority, provenance, reproducibility and an accountable owner live in four different places, and the gaps between them are where failure walks through.
Where it lands
Govern the decision, by its blast radius, whatever is under the hood. The work is making sure the controls you need exist for the decisions you have already let the machine make. The model, the label and the fear are detail.
If you cannot yet answer what your systems are allowed to decide, what happens when they are wrong, or who answers for them, that is the conversation worth having now. An AI diagnostic starts there.
Series
AI governance- 1. The “AI” premium: what buyers overpay for
- 2. Govern the decision, not the architecture
- 3. When you can’t govern what you should
- 4. Cyber insurance wasn’t built for AI. It’s being rewritten right now.
- 5. Your AI model isn’t the risk. What you let it decide is.
