Governing AI When the Stakes Are Real

Introduction

AI governance frameworks are, at their core, how humans design themselves into the loop, how we decide what authority to delegate to machines, and how we retain meaningful control as those systems become more capable. Governance exists to make power legible, bounded, and accountable.

To date, most AI governance approaches borrow heavily from models built for traditional software and cybersecurity, generally known as “task-based governance models”. They focus on controlling access and execution: what data a system can see, which tools it can use, and which actions it is technically permitted to perform. This approach works well for static systems that behave predictably, where humans retain full decision authority and software simply carries out instructions.

But AI is fundamentally different. Modern AI systems do not just execute tasks; they interpret context, weigh signals, and make trade-offs. Governing them purely through system permissions treats a reasoning system like legacy software from two decades ago. It assumes that control over inputs and tools is sufficient, even as the AI system’s real impact comes from how it judges and decides.

This mismatch creates serious problems. Task-based governance leads to opaque systems where decisions cannot be explained, bias cannot be isolated, and accountability is difficult to assign. It produces black boxes at precisely the moment when transparency matters most. Applying governance frameworks designed for firewalls and access controls to AI decision-making systems does not manage risk—it amplifies it and creates a fertile field for bias and opaqueness.

This paper proposes a different approach: the AI Decision Framework. Rather than governing solely what AI can technically do, the framework governs what kinds of decisions AI is allowed to make, at what level of autonomy, and with what human oversight. Low-consequence AI decisions can be handled quickly and automatically. High-consequence decisions require structured partnership with a human, clear review points, and an explicit record of reasoning. This mirrors how we already govern judgment in society, from medical training to child welfare to public service, because it is intuitive, human-centered, and fair by design. It preserves speed where speed is safe, introduces friction where consequences demand care, and keeps bias visible rather than buried. Most importantly, it creates a governance foundation that allows AI to scale responsibly, serving the full breadth of society, not just the systems that are easiest to automate.

The call to action is simple: stop treating AI like a collection of tools, and start governing it like a decision-maker. The systems are already moving in that direction. Governance needs to catch up, now.

Decision autonomy and AI

The AI Decision Framework starts from a simple premise: not all decisions are equal, and AI should not be given the same level of autonomy in every situation. The framework therefore begins by classifying AI-supported decisions into four levels of autonomy, based on impact and how consequential a decision is for individuals, organizations, or society at large. As the potential impact of a decision increases, so does the level of required human oversight.

This approach brings proportionality to AI governance. Instead of treating all AI actions as interchangeable technical events, it explicitly aligns autonomy with consequence. Routine, low-impact decisions can be handled quickly and automatically, while decisions with meaningful human, financial, or societal implications are intentionally slowed, reviewed, and shared with human decision-makers.

By doing so, the framework makes human involvement explicit rather than assumed. It is designed into the system itself, with clear expectations about when humans must review, approve, or fully own a decision. This clarity reduces ambiguity for teams building AI systems and for those responsible for governing them.

Equally important, the framework creates a shared language. Engineers, product leaders, executives, government officials, regulators, and external stakeholders can all understand how judgment is being exercised and why certain decisions require more care than others. “Human in the loop” is an operating model rather than a slogan.

A necessary shift

AI is not simply executing instructions; it is participating in judgment. Yet task-based governance asks the wrong question. It asks what the system touched, rather than what decision it influenced, and who bears the consequences when that decision goes wrong. The result is a dangerous mismatch: systems that shape access, opportunity, safety, and outcomes, governed as if they were static tools.

Bias in AI rarely comes from a single bad actor or intentional harm. It emerges when systems make trade-offs quietly, without visibility into whose values were encoded, whose data was prioritized, and whose experiences were left out.

This matters deeply for equity. When governance stops at tasks, those trade-offs remain hidden. Bias becomes harder to detect, harder to challenge, and nearly impossible to correct, especially for communities that already have the least power.

By governing decisions rather than just technology, the AI Decision Framework keeps bias visible. It creates clear moments where humans must review outcomes, understand reasoning, and take responsibility. It makes it possible to ask not only what an AI decided, but why, and whether that decision aligns with shared values of fairness, inclusion, and dignity.

This mirrors how judgment already works in the real world. Junior doctors do not make the same calls as senior physicians. Social workers treat routine cases differently from situations involving a child’s safety. Financial institutions scrutinize high-stakes decisions more carefully than low-risk ones. These structures exist because society understands that unchecked authority, human or machine, creates harm.

The moment to make this shift is now. As AI systems gain greater influence over lives and livelihoods, continuing to govern them as if they were ordinary software is not neutral. It is a choice, and a risky one. If we want AI to truly serve the full breadth of society, we must move immediately from task-based control to decision-based governance, and design human judgment, equity, and accountability into the system from the start.

About the author

Simone Garreau is the co-founder and CEO of Prism Layer, an AI-native risk intelligence platform that turns a company’s own risk framework into an execution layer, automatically pulling data, applying guardrails, and producing clear, defensible risk decisions. Prism Layer was born out of her firsthand experience leading risk teams in environments where speed, innovation, and accountability all had to coexist.

Over the past decade, Simone has built and scaled enterprise risk programs at Western Union, Robinhood, Remitly, and Block, working closely with boards, regulators, and product leaders during moments of rapid growth, crisis, and transformation. Earlier in her career, Simone worked at the Bill & Melinda Gates Foundation on malaria eradication and policy, experiences that shaped her conviction that systems must be designed around human outcomes, not technical abstraction. That throughline, human-centered governance at scale, has guided her work across financial services, and now underpins her thinking on how AI should be governed as it takes on greater decision-making authority.