Run-Time AI Governance: Why Your AI Policies Don't Survive Production

Run-time AI governance is the practice of observing, evaluating, enforcing and evidencing the behaviour of an AI system while it is actually running in production, rather than only documenting how it is supposed to behave before it goes live. It is the difference between a policy that says an AI system must not leak personal data and a control that inspects every response in real time and blocks the one that does. Most enterprise AI governance today is the former. Almost all enterprise AI risk lives in the latter.

This matters because of a gap that very few governance programmes have closed, and that becomes wider and more dangerous the moment AI moves from assisting people to acting on their behalf. We call it the governance-execution gap: the distance between what your governance documents assert and what your AI systems actually do once they meet real data, real users, and real adversaries. Design-time governance writes the assertion. Run-time governance is what makes it true.

Design-time governance versus run-time governance

It helps to separate the two clearly, because most organisations have invested heavily in one and barely at all in the other.

Design-time governance is everything you do before a system is in production. It is the AI policy, the risk assessment, the impact assessment, the model selection rationale, the approval workflow, the sign-off. It is essential, and it is where frameworks such as ISO/IEC 42001 and the EU AI Act place much of their visible weight. Done well, it ensures a system is pointed at the right problem, within agreed boundaries, with an accountable owner. It is necessary. It is also, on its own, insufficient, because it describes intended behaviour at a single moment and then the system goes off and behaves for months or years without that document watching.

Run-time governance is everything that happens after deployment, continuously, for as long as the system is live. It is the monitoring that detects when a model's accuracy drifts as the world changes underneath it. It is the guardrail that inspects a prompt for an injection attack or a response for toxic content, personal data, or a hallucinated instruction, and acts before the output reaches a user. It is the behavioural monitoring that notices an autonomous agent attempting an action outside its approved scope. It is the audit trail that records what actually happened, so that an incident can be reconstructed and a regulator can be answered.

The reason policies do not survive production is simple. A policy is a static statement about a dynamic system. The model drifts. The data shifts. The users probe. The adversaries adapt. Agents chain decisions in combinations nobody anticipated at sign-off. None of that is visible to a document sitting in a governance repository. It is only visible at run-time, to controls that are watching the system behave.

Why run-time governance has become unavoidable

For years, the documentary approach was almost defensible, because AI mostly advised and people mostly decided. A human read the model's output and exercised judgement, and that human was the run-time control, a slow, expensive, but real check on every consequential action. Three shifts have removed that comfort.

The first is generative AI, whose outputs are open-ended and non-deterministic. A traditional model produced a score you could threshold. A large language model produces free text that can be fluent, confident, and wrong, can leak data it was never meant to surface, and can be steered by a cleverly worded prompt. You cannot govern that with a pre-deployment checklist. You govern it by inspecting inputs and outputs as they flow.

The second is autonomy. Agentic systems perceive, plan, call tools, and act across multiple systems, often chaining several decisions before any human sees a result. The human-in-the-loop control that quietly underpinned the old model has gone. Gartner expects that by the end of 2026 around 40 percent of enterprise applications will embed task-specific agents, up from under 5 percent in 2025, and predicts that by 2027 roughly 40 percent of enterprises will demote or decommission agents because of governance gaps surfaced only after a production incident. Read plainly: the governance was found wanting at run-time, after the agent had already acted.

The third is the regulatory direction of travel, which increasingly asks for evidence of continuous control rather than a one-off attestation. The EU AI Act requires that high-risk systems enable effective human oversight and maintain logs, both of which are run-time properties. UK financial regulators, applying existing rules to AI, expect operational resilience and ongoing monitoring of outcomes, not a policy filed at launch. A document cannot satisfy a requirement that is fundamentally about what the system keeps doing.

What run-time AI governance actually consists of

Strip the vendor language away and run-time governance reduces to four functions that operate in a continuous loop. We use them as the practical spine of every run-time programme we build.

Observe

You cannot govern what you cannot see, and in production that means telemetry on the system's actual behaviour: inputs, outputs, tool calls, latency, cost, and the decisions an agent takes. Observability is the foundation, because every other function depends on it. For predictive models this includes accuracy and fairness drift; for generative systems it extends to hallucination rates, toxicity, and prompt-injection attempts; for agents it means tracing a full chain of actions across the tools and systems the agent touches, at the conversation, interaction, and individual tool level. Without this, governance is blind the instant the system leaves the lab.

Evaluate

Raw telemetry is noise until it is judged against a standard. Evaluation is the continuous, often automated, assessment of behaviour against thresholds you set: is the model still accurate enough, fair enough, fast enough; is this output safe to return; is this action within scope. The mature pattern is in-the-loop evaluation, where outputs and actions are scored as they happen rather than sampled weeks later in a review. This is where development-time test suites become production controls: the evaluations you ran to validate a system before launch should run continuously against it after launch, because a system that passed last quarter can fail today.

Enforce

Observation and evaluation without enforcement is a dashboard, not a control. Enforcement is the ability to act on a judgement in real time: to block a non-compliant output, redact sensitive data before it is returned, route a low-confidence response to a safer path or a human, halt an agent that crosses a threshold, and roll back cleanly when something goes wrong. The important design principle, and one Gartner has been explicit about for agents, is that enforcement must be graduated rather than binary. A system that drafts internal text and a system that can move money do not warrant the same clamp. Matching the strength of enforcement to the autonomy and access of each system is what lets you move fast where it is safe and intervene hard only where it matters.

Evidence

Finally, every observation, evaluation, and enforcement action must be recorded, because the value of run-time governance is realised twice: once when it prevents harm, and again when it proves, to an auditor, a regulator, a board, or a customer, that the system has been under control all along. A continuous, tamper-evident record of what each system did, why a particular output was blocked, who owns the system, and how it has performed over time is what turns governance from a claim into demonstrable fact. This is also precisely what the evidence-based frameworks demand: ISO 42001's documented information, the EU AI Act's logging and record-keeping, and a UK regulator's expectation that you can show your work.

Observe, evaluate, enforce, evidence. The first draft of this thinking is the same alignment-control-evidence discipline that underpins good design-time governance, extended into production and made continuous. The principles do not change when the system goes live. The tempo does, from a one-off sign-off to a permanent loop.

Run-time governance and the UK regulated enterprise

For UK enterprises, and especially regulated ones, run-time governance is not an optional refinement. It is increasingly the only way to meet expectations that existing regulators are already applying to AI.

The UK has deliberately declined to pass a single AI Act, governing instead through existing sector regulators applying their existing remits. In financial services this is consequential. The FCA's Consumer Duty requires firms to deliver and monitor good outcomes for customers on an ongoing basis, which for an AI-driven decision means watching the outcomes the model produces in production, not certifying its design once. The PRA's model-risk expectations under SS1/23 push regulated banks toward continuous model monitoring and validation. Operational resilience rules expect firms to understand and maintain important business services, which now frequently sit on top of AI. The ICO's guidance on automated decision-making focuses on decisions with legal or similarly significant effects on people, the kind of decisions that are made repeatedly, at run-time, long after any impact assessment was signed.

None of these are satisfied by a design-time artifact. They are satisfied by a system that is observed, evaluated, enforced, and evidenced while it runs. And for organisations with any exposure to the EU market, the EU AI Act's logging and human-oversight requirements pull in the same direction, regardless of the UK's lighter statutory touch. The practical conclusion for a UK enterprise is that run-time governance is where the regulatory expectations of both jurisdictions actually converge.

How watsonx.governance implements run-time governance

We are an IBM Gold partner and implement run-time governance on IBM watsonx.governance, so it is worth being concrete about how the four functions map to real tooling, because the gap between governance intent and a working control is exactly where most programmes stall.

For observe and evaluate, watsonx.governance uses Watson OpenScale to monitor deployed models against thresholds you set, for fairness, drift, and quality, and extends this to generative systems with preset thresholds on inputs and outputs that detect toxic language, hate speech, and other unsafe content in real time, alongside metrics for latency and throughput. For agentic systems specifically, Agent Monitoring and Insights tracks an agent's decisions, behaviours, and performance in production and raises alerts when metrics cross thresholds, with evaluation aggregated at the conversation, interaction, and tool level so you can see not just that an agent misbehaved but where in its chain it did.

For enforce, the platform supports the graduated pattern run-time governance requires: continuous in-the-loop evaluation with policy enforcement, and automated block, route, or fallback when an output or action breaches policy, including dynamic routing that can divert to a safer path when contextual quality is low. Guardium AI Security brings live security posture, vulnerability and prompt-injection detection, into the same governance console, so a procurement-style approval and a real-time security signal sit in one place rather than two disconnected workstreams.

For evidence, AI Factsheets act as automatically generated records, IBM describes them as nutritional labels for models, capturing metadata and an audit trail across the lifecycle, from the rationale for choosing a model through to its behaviour in production, exportable for audits. The Governance Graph maps the whole AI estate, tracing what AI is in use, for what purpose, under what controls, and whether those controls are working, across platforms and providers. Integration with OpenPages connects AI risk to enterprise risk and compliance workflows. IBM was named a Leader in the 2026 Gartner Magic Quadrant for AI Governance Platforms, which matters less as a badge than as confirmation that the run-time capability is enterprise-grade.

The tooling is not the governance. A platform that can observe, evaluate, enforce, and evidence still has to be configured against a framework the organisation has actually built: the right thresholds, the right enforcement tiers, the right ownership, the right evidence requirements. That implementation, turning the capability into a working control plane tied to real policies and real regulatory obligations, is the work. The platform makes it possible. It does not make it automatic.

Where to start

If your AI governance today is mostly design-time, the move to run-time governance does not begin with a tool purchase. It begins with three honest questions about your most consequential AI systems. Can you see what they are doing in production right now, or only what you intended them to do at sign-off? If one of them produced a harmful or non-compliant output this afternoon, would anything catch it before it reached a user? And if a regulator asked you to prove that system has been under control for the last six months, could you, with evidence rather than assertion?

For most organisations the answers are uncomfortable, and the discomfort is the point. The governance-execution gap is invisible until you look for it, and it is widening as AI becomes more generative, more autonomous, and more deeply embedded in the decisions that regulators care about. Closing it is the work of run-time governance: observe, evaluate, enforce, evidence, continuously, for as long as the system is live. That is where AI risk actually is, and therefore where governance has to actually be.

Frequently asked questions

What is the difference between design-time and run-time AI governance?Design-time governance is the policy, risk assessment, and approval work done before an AI system goes live. Run-time governance is the continuous observation, evaluation, enforcement, and evidencing of the system's behaviour after it is in production. Design-time describes intended behaviour; run-time governs actual behaviour. Both are needed, but AI risk is realised at run-time.

Are AI guardrails the same as run-time governance?Guardrails are one part of it. Guardrails typically inspect and act on inputs and outputs, blocking, redacting, or routing in real time. Run-time governance is broader: it also includes continuous monitoring for drift and quality, behavioural monitoring of agents, graduated enforcement, and the evidence trail that proves control over time.

Does run-time AI governance add latency?It can, which is why enforcement is designed in tiers. Lightweight checks run inline within the request budget for most systems, while heavier evaluation can run asynchronously or be reserved for higher-risk actions. The aim is enforcement proportionate to risk, not maximum friction everywhere.

How does run-time governance help with the EU AI Act and UK regulators?The EU AI Act's logging and human-oversight requirements and UK regulators' expectations of ongoing outcome monitoring and operational resilience are run-time properties by nature. They ask what a system keeps doing, which only continuous monitoring and evidence can answer.

Can we do run-time governance on AI we did not build?Yes. Much enterprise AI is procured or third-party. Platforms such as watsonx.governance are designed to govern models, applications, and agents across multiple providers, so run-time governance applies to AI you consume as well as AI you build.

Aligne is an AI governance advisory, consulting and implementation partner for UK enterprises, an IBM Gold partner, and a specialist in run-time AI governance. If you want to find your governance-execution gap before an incident does, that is the work we do.