A prompt policy that lives in a wiki, a client-side helper, or a single application repository is not an enforced policy. The first unreviewed integration, provider migration, or internal prototype can bypass it. To enforce prompt policies consistently, teams need controls at the point every model request must cross: one governed doorway between applications and model providers.
That changes the operational question. Rather than asking every product team to remember which data may be sent to which model, platform teams can apply the same controls to each request before it reaches OpenAI, Google Gemini, Anthropic Claude, DeepSeek, Mistral, or Cohere. The application keeps its business logic. The control plane makes the decision.
Why application-level policies do not hold up in production
Most LLM estates become fragmented quickly. A customer-facing assistant uses one provider. An internal knowledge workflow uses another. A data team runs batch enrichment with a third. Each integration may have its own environment variables, logging choices, fallback behaviour, and interpretation of security guidance.
That fragmentation creates predictable gaps. A team might add prompt-injection checks to its newest assistant but not to a legacy workflow. Another might redact email addresses before calling one provider while a separate service logs full prompts for debugging. Finance may see a growing provider bill without knowing which route, application, or feature caused it.
Policies written into application code also introduce an ownership problem. Security and compliance teams define the control. Product engineers implement it differently across repositories. When the policy changes, every team must schedule and deploy an update. That is slow when the required response is immediate, such as blocking a newly identified prompt-injection pattern or tightening treatment of sensitive data.
A gateway is useful because it separates these responsibilities. Applications request a capable model. The platform team defines the route, the controls, the permitted provider choices, and the evidence retained for review.
Start with policies that can make a request-level decision
A usable prompt policy needs an action, not just a principle. “Protect personal data” is a good objective, but it does not tell infrastructure what to do when a prompt contains a phone number, a bank detail, or a customer record.
For each policy, define the scope, detection condition, action, owner, and expected evidence. The action should be specific: log, warn, redact, block, or moderate. The owner should be able to change the rule without waiting for every consuming application to alter its own middleware.
Three policy families normally matter first.
Protect the input before provider selection
Prompt injection and jailbreak attempts are not only a chatbot problem. They can affect internal copilots, document-processing flows, support automation, and agents using retrieved content. An instruction hidden inside a document can attempt to override the application’s intended task, extract confidential context, or influence downstream tool use.
A prompt shield assesses these attempts before the request is routed. This placement matters. A suspicious request should not be sent to a different model merely because a route has a lower price or better current availability. It should be evaluated and handled according to the policy first.
The goal is not to claim that any detection system is perfect. Attack patterns change, and the right posture depends on the workflow. A public assistant may need a firm block policy. A research workflow may require warnings and review so analysts can inspect unusual inputs. What matters is that the chosen control is applied consistently, not selectively.
Apply DLP to data that must not leave the boundary
DLP turns data-handling rules into an execution decision. Depending on the policy, sensitive values can be logged, warned on, redacted, or blocked. Redaction replaces detected material with [REDACTED]; it does not create a reversible placeholder or restore a value later.
This distinction is operationally useful. Redaction may be appropriate when a prompt can still be answered without the sensitive field, such as a request to improve the wording of a message. Blocking is usually the safer decision when the value is central to the task or where disclosure is prohibited.
Teams should avoid treating all data alike. A blanket block rule can break legitimate workflows and drive engineers towards ungoverned workarounds. A permissive rule can expose more than intended. Start with the categories that carry the highest regulatory, contractual, or commercial impact, then test the effect on real request samples.
Moderate the result, not only the prompt
A clean input does not guarantee an acceptable output. The model can generate harmful, unsuitable, or policy-violating material in response to otherwise ordinary requests. Output moderation provides a final control point before the response returns to the calling application.
Input and output controls solve different problems. Prompt shielding addresses hostile or manipulative instructions. DLP governs sensitive material in the request. Output moderation addresses what the model produces. Combining them creates a clearer decision path than relying on a single, catch-all rule.
Put enforcement in the correct order
Control order is part of the policy. If routing happens before inspection, a request may already have been sent to a provider before the platform determines it should have been blocked or redacted.
The required sequence is straightforward: prompt shield, then DLP, then routing, then output moderation. This gives security controls precedence over price and model-selection decisions, while still allowing routing to choose the most cost-effective capable model for requests that pass the input policies.
That structure also makes incident analysis less ambiguous. A trace can show whether the request was stopped by the prompt shield, changed by DLP, routed to a provider, or moderated on the way out. Without an ordered control path, teams often have logs but not an explanation.
Deploy the gateway without rewriting the application
For an application already using an OpenAI-compatible client, adoption can be limited to the API key and base URL. The request shape and business logic stay byte-for-byte identical.
import os from openai import OpenAI
client = OpenAI( api_key=os.environ["ROUTEUR_KEY"], base_url="https://api.routeur.ai/v1" )
response = client.chat.completions.create( model="auto", messages=[ {"role": "user", "content": "Summarise this customer request."} ] )With model="auto", routing rules and priorities can select an appropriate provider and model after the request clears the input controls. Where a workload needs a defined operating profile, the application can use a plain logical route name instead. This is useful when teams need predictable model eligibility for a regulated process, a high-volume classification task, or a premium customer feature.
Scoped API keys add another boundary. A key can represent the application or team using it, making it possible to contain access and attribute consumption without embedding governance logic in every service. Named routes then keep provider and model decisions under platform ownership rather than scattering them across codebases.
routeur.ai provides this governed layer through an OpenAI-compatible endpoint, so teams can introduce central enforcement without a new SDK or an application rewrite.
Test the policy before making it disruptive
The safest rollout is progressive. Begin by observing representative traffic and reviewing the cases that would trigger a rule. Use dry-run testing where appropriate to understand the policy outcome without applying the production action. This is particularly valuable for DLP rules, where false positives can reveal fields that are not sensitive in a given business context.
Then move in stages. A warning period lets product and security teams see the operational impact. Redaction can protect data while preserving some workflows. Blocking should follow where the risk demands it and where the failure behaviour is understood by the application owner.
Test more than obvious malicious prompts. Include retrieved documents, copied support tickets, multilingual requests, malformed inputs, long prompts, and prompts containing data that resembles a sensitive identifier but is not one. A policy that only works on clean test strings is not ready for production traffic.
Make enforcement visible to engineering, security and FinOps
Enforcement without evidence becomes hard to defend. Per-request traces should provide the operational record: the policy result, route decision, provider and model outcome, latency, and cost-related context. Metadata-only logging by default reduces unnecessary payload exposure; payload retention should be an explicit opt-in decision.
This design is especially relevant for UK and EU organisations balancing observability with data minimisation. Audit trails can support the record-keeping expected of governed AI operations, but teams should still define retention practices around their own obligations and risk profile. More data in a log is not automatically better evidence.
Cost controls belong in the same operating model. A hard spend cap can alert or block further requests with a 429 response when the limit is reached. It should not be treated as a routing preference or an automatic way to move traffic elsewhere. The application needs a defined response for that condition, whether it is a user-facing message, deferred work, or an approved fallback process.
Treat policies as production configuration
Prompt policy enforcement is not a one-time security project. New applications, new models, changing regulations, and revised product features will all test the original assumptions. Review traces, blocked requests, warnings, spend patterns, and exceptions regularly. Tighten policies where the evidence shows unnecessary exposure, and relax only where a controlled business need is clear.
The practical standard is simple: every model request should receive the same considered treatment, regardless of which team built the feature or which provider eventually serves it. When that control sits at one governed doorway, security, compliance, engineering, and FinOps can work from the same request record rather than four competing versions of the truth.