A production AI governance case study is rarely about choosing a single best model. It is about what happens when dozens of applications, teams and use cases begin sending sensitive business context to multiple providers - each with different pricing, availability and data-handling considerations.
Consider a regulated software company with an internal support copilot, a customer-facing document assistant and automated back-office workflows. The engineering team has integrated LLMs successfully. The harder problem arrives afterwards: finance cannot reliably attribute spend, security cannot see which requests contained personal data, and compliance needs evidence that controls were applied at the moment each request was made.
The company does not need another application rewrite. It needs one governed doorway between its applications and model providers.
The operating problem was not model quality
At first, each product team used provider SDKs directly. That was expedient. It also meant routing decisions were embedded in application code, credentials were spread across services, and safety controls varied by team.
The document assistant used a high-capability model for every task, including straightforward classification and extraction. The support copilot occasionally received pasted customer account details. A workflow team retained prompts in debugging logs because those logs were the only available way to investigate unexpected outputs. None of these decisions were unreasonable in isolation. Together, they created an operating model with three material gaps.
First, the company had no consistent request-level policy boundary. A sensitive prompt might be handled carefully in one service and passed through unexamined in another. Secondly, cost grew with usage but had no central hard limit. Finally, producing an audit trail meant gathering partial records from applications and provider consoles after an incident or review.
Adding a security library to each codebase would have addressed part of the problem, but it would also have created a maintenance programme. Every new service, provider and model change would require another implementation cycle. The company needed control to be centralised, while keeping product delivery moving.
AI governance case study architecture
The practical change was to place a governed API control plane between applications and providers. Applications continued to make OpenAI-compatible requests. Rather than choosing a provider endpoint directly, they sent requests through a single gateway configured with routes, routing rules and policy controls.
The request path mattered. Each request passed through prompt-injection and jailbreak shielding first, then DLP inspection, then routing, then output moderation. This order prevents routing economics from becoming a reason to bypass protective checks. It also means the organisation can apply the same baseline controls whether the request ultimately reaches OpenAI, Google Gemini, DeepSeek, Anthropic Claude, Mistral or Cohere.
For this illustrative deployment, the platform team established named routes for distinct risk and performance needs. A general route could select the most cost-effective capable model. A restricted route could constrain provider choices for a workload with tighter requirements. Routing priorities and automatic failover reduced dependence on a single provider without forcing application teams to implement their own fallback logic.
The application change was deliberately small:
import os from openai import OpenAI
client = OpenAI( api_key=os.environ["ROUTEUR_KEY"], base_url="https://api.routeur.ai/v1" )
response = client.chat.completions.create( model="auto", messages=[ {"role": "user", "content": "Summarise this support request."} ] )For teams already using an OpenAI-compatible client, the business logic stays byte-for-byte identical. The operational change is the API key and base URL. That distinction is central: governance adopted through a long rewrite programme is often governance deferred.
Policies were made explicit at the request boundary
The security team began with policies it could explain in plain operational terms. Prompts suspected of attempting to override instructions or extract protected behaviour were stopped by the prompt shield. DLP controls inspected for personal data and could log, warn, redact or block according to the route and workload.
Redaction was particularly useful for lower-risk internal workflows where the request could proceed without exposing a detected value to the provider. The resulting prompt used [REDACTED]; there was no attempt to tokenise a value and restore it later. For higher-risk workflows, the appropriate action was blocking rather than redaction. The right choice depends on whether the task can still produce a reliable result after sensitive context has been removed.
Output moderation closed the other side of the transaction. A safe prompt does not guarantee a suitable response, especially where users can introduce unpredictable content. Applying moderation after the model response gave the company a consistent final check before content reached an employee or customer.
This did introduce trade-offs. More restrictive DLP rules can increase false positives, and blocking a request may interrupt a valid workflow. The company therefore used warnings during initial observation where appropriate, reviewed trace patterns, and then tightened actions for use cases where exposure carried a higher consequence. Governance is not a switch marked “on”. It is a set of decisions that should become more precise as production evidence accumulates.
Spend control became an engineering control
Before centralisation, monthly model costs were visible only after invoices arrived. That encouraged retrospective conversations: why did usage rise, which service drove it, and was the increase justified?
With a central control plane, the company applied hard spend caps and budgets to make cost a live operating constraint. A cap could alert or block further requests with a 429 when reached. It did not silently reroute traffic to a cheaper model, because a budget rule and a quality-routing decision are different controls with different owners.
The routing layer addressed efficiency separately. It selected the most cost-effective capable model for each request, rather than assuming that the highest-priced model was required for every task. In production, that can materially reduce waste for high-volume, repeatable workloads. routeur.ai reports around 20% average savings, but the realised result depends on prompt complexity, the provider mix, route configuration and how much traffic can safely use lower-cost capable models.
This separation gave FinOps and engineering a cleaner conversation. FinOps could define budget boundaries. Platform engineers could tune routes and priorities. Product teams could measure whether output quality still met the standard for their specific workflow.
Audit evidence became part of the normal path
The compliance team did not need a replay of every prompt by default. It needed a defensible record of how requests were handled: which route was used, what policy action occurred, where the request went, what it cost, and whether fallback or moderation was involved.
Per-request traces provided that operational evidence. Metadata-only logging by default reduced unnecessary payload retention, while explicit opt-in payload retention remained available for teams that had a justified debugging or quality-review need. This is a meaningful design choice in regulated environments. Retaining everything “just in case” can create a separate data-protection problem.
The trace record also changed incident response. Instead of asking multiple teams to reconstruct a request path from application logs, the platform team could examine the request-level decision trail. Scoped API keys supported clearer ownership boundaries, while named routes made policy intent visible rather than hidden in application branches.
For a UK or EU organisation, this structure also provides useful support for GDPR processor obligations and EU AI Act record-keeping. It is not a substitute for legal assessment, data classification or organisational accountability. It does, however, turn governance from a collection of stated policies into controls and records that operate where model requests actually flow.
What the company would measure next
The success criteria were intentionally operational. The platform team tracked policy blocks and warnings by application, DLP detections by route, model and provider selection, failover events, cost per workflow, and the proportion of requests using each route. Product teams paired those figures with quality checks and user feedback.
A rise in blocked prompts could indicate a real security issue, a poorly designed workflow or an over-sensitive rule. More traffic on a lower-cost model may be a saving, or it may warrant closer quality evaluation. Metrics only become governance when somebody has authority to interpret them and change the relevant rule.
That is the useful lesson from this AI governance case study: production control does not come from asking every team to be more careful. It comes from putting the controls, evidence and economic decisions in the request path - where every application must pass through them. The most effective next step is usually not a broad transformation programme, but a single workload routed through one governed doorway and measured properly from its first request.