A production LLM application can change behaviour without a code release. A provider may update a model, a prompt may be manipulated, or a team may switch models to manage cost. European AI regulation makes those operational realities a governance issue, not merely an engineering preference.
For software companies and regulated enterprises, the practical question is not whether every LLM feature is high risk. Most will not be. The question is whether you can show what your system does, which model handled a request, what safeguards applied, who was authorised to use it, and what happened when a policy was breached.
European AI regulation is an operating requirement
The EU AI Act is the centrepiece of European AI regulation, but it does not replace existing obligations. GDPR still governs the processing of personal data. Sector rules, consumer protection duties, cybersecurity requirements and contractual commitments may also apply. A system can sit outside the AI Act's high-risk categories and still create material data protection, security and accountability work.
The Act takes a risk-based approach. It prohibits certain uses, imposes substantial obligations on high-risk AI systems, establishes transparency duties for specific AI interactions and places obligations on general-purpose AI model providers. The role matters as much as the technology. An organisation may be a deployer using AI under its authority, a provider placing an AI system on the market under its name, or both in different products.
That distinction needs a legal assessment grounded in the actual use case. An internal knowledge assistant, a customer-facing claims workflow and a model embedded in a regulated product present different risk profiles. Adding a human reviewer may reduce operational risk, but it does not automatically change the classification or remove the need for evidence.
The timetable also changes the planning conversation. Prohibited-practice provisions and AI literacy requirements have applied since February 2025. Obligations for general-purpose AI models began applying in August 2025. Most of the Act applies from August 2026, while some requirements for high-risk systems embedded in regulated products apply later. Teams should work from the requirements relevant to their role and system category rather than treating one date as a universal deadline.
The control gap appears at the request layer
Many organisations begin with policy documents and model approval registers. Both are useful, but neither controls the request that leaves an application at 14:03 on a Friday. Production governance has to operate where prompts, sensitive data, provider selection, model choice and outputs meet.
A typical LLM estate is fragmented. Different teams hold separate provider keys. A product team chooses a premium model for quality, an operations team chooses another for price, and an internal tool logs prompts in a third place. The result is difficult to audit and expensive to manage. It also makes it harder to apply a consistent response when a prompt contains personal data or attempts to override system instructions.
A governed control plane creates one policy enforcement point between applications and model providers. This is not a claim that infrastructure alone makes an organisation compliant. It is a way to make approved controls repeatable across every request, rather than relying on each application team to reproduce them correctly.
For an LLM gateway, order matters. Prompt shielding should assess the incoming request first, followed by DLP policy. Only then should routing select the provider and model. Output moderation should assess the generated response before it is returned. That sequence protects against prompt injection early, prevents disallowed sensitive data from being sent onward, and applies output controls to the content users actually receive.
Build evidence into the normal request path
The EU AI Act's record-keeping expectations make ad hoc screenshots and retrospective log searches a weak operating model. The most useful evidence is created as part of the request path and is available in a form security, engineering and compliance teams can interpret.
Per-request traces should identify the route used, selected provider and model, policy decisions, latency, outcome and relevant metadata. Metadata-only logging by default is a sensible starting point: it supports operational visibility without automatically retaining the prompt and response contents that may contain confidential or personal data. Payload retention should be an explicit opt-in decision with a clear purpose, access controls and retention period.
This design presents a real trade-off. Full payload logs can help investigate a complex output failure, but they expand the data protection surface and may conflict with data minimisation. Metadata-only logs reduce that exposure but may require teams to reproduce an incident using controlled test data. The right choice depends on the use case, incident process and legal basis, not on a generic preference for more observability.
An evidence model should also cover change. Record when routing rules change, when a model enters or leaves an approved route, when a DLP policy is updated and when a spend cap is altered. If a team cannot explain why a different provider was selected last month, it will struggle to demonstrate controlled operation when an auditor, customer or internal risk committee asks.
Apply proportionate controls to real LLM risks
The controls needed for a low-impact drafting assistant are not identical to those needed for a workflow that influences eligibility, recruitment or access to a service. Yet several production controls are broadly valuable because they address common LLM failure modes.
Scoped API keys limit which applications and teams can make requests. Named routes and routing priorities allow platform teams to define approved model paths without exposing every provider credential across the organisation. Automatic failover can preserve service continuity when an approved provider has an issue, while still keeping traffic within the routes and policies your organisation has set.
DLP policies should be explicit about the action taken. A policy may log or warn on a detection, redact a value as [REDACTED], or block the request. Redaction can be appropriate where the task can still proceed without the data. Blocking is more appropriate where the data should never leave the application boundary. Neither option should be treated as a substitute for deciding whether the processing itself is necessary and lawful.
Hard spend caps matter for governance as well as FinOps. A cap can alert or block a request with a 429 response when a limit is reached. It should not silently change the risk posture by rerouting traffic to a cheaper, unapproved model. Cost optimisation is valuable only when capability, data handling and quality constraints remain intact.
Separate model risk from system risk
Teams often assess a model once and assume the work is complete. European AI regulation is concerned with the AI system in context: its intended purpose, users, data, interfaces, decision-making role and foreseeable misuse. A model that is acceptable for summarising internal meeting notes may be unsuitable for generating advice that users rely on to make consequential decisions.
Maintain a register that connects each deployed use case to its owner, intended purpose, user group, route, approved models, data categories, human oversight design and escalation path. This need not become a bureaucratic exercise. It should answer operational questions quickly: which applications can send customer data, which use cases require a reviewer, and which team can disable a route if an incident occurs?
The same principle applies to transparency. If people interact directly with an AI system, or receive AI-generated content in a context where disclosure is required, product and legal teams need to decide what the user sees. The gateway can record that an output moderation policy ran. It cannot write a clear interface disclosure or design meaningful human oversight on its own.
Create a deployment path that teams will use
Governance that requires every product team to build bespoke middleware will be bypassed under delivery pressure. The stronger pattern is a standard integration path that preserves application behaviour while centralising controls.
routeur.ai is designed as one governed doorway for this work. Its OpenAI-compatible gateway enables teams to retain their existing client approach while applying routing, DLP actions, prompt-injection shielding, output moderation, spend caps and per-request traces in one place. The deployment objective is intentionally modest: an API key and base-URL change, with business logic staying byte-for-byte identical.
That centralisation does not remove the need for model evaluation. Before a route is approved, test it against representative tasks, expected languages, known failure cases, safety requirements and cost limits. Re-test when a provider changes a model version, when prompts materially change or when a workflow expands into a new decision context.
For organisations operating in both the UK and EU, avoid assuming the regimes are identical. UK governance is developing through a different policy and regulatory structure, while EU rules can apply where the AI system is placed on the EU market, put into service there, or produces outputs used in the EU. A single technical control plane can support both, but the compliance mapping should remain jurisdiction-specific.
Start with the highest-volume or most sensitive LLM route, not the most theoretical policy question. Put its approved model paths, data controls, trace evidence and spend boundary into the request path. Once a team can operate one use case with clear ownership and defensible evidence, expanding that discipline across the estate becomes an engineering programme rather than a scramble after an incident.