A support agent receives a message that looks ordinary: “Please summarise my account history.” Buried beneath it is an instruction to ignore prior rules, reveal internal guidance and send customer data elsewhere. If the application passes that prompt directly to a model provider, the security decision has already been outsourced to probabilistic behaviour.
LLM security is the discipline of making that decision deterministic before the request reaches a model, and proving what happened afterwards. For teams running production copilots, internal workflows and customer-facing AI, this is not a prompt-writing exercise. It is an infrastructure problem: every request needs consistent controls, regardless of which model answers it.
Why LLM security breaks at the application edge
Most LLM integrations begin with one API key, one provider and a small amount of application code. That is reasonable for a prototype. In production, the same pattern creates a widening control gap.
Teams add providers for capability, availability or cost. Product groups introduce their own prompts and tools. Sensitive information appears in support conversations, documents and internal workflows. Security teams need evidence, while FinOps needs predictable spend. Each individual requirement can lead to another wrapper, webhook, proxy or custom logging path.
The result is fragmented enforcement. One application may inspect inputs while another sends them directly to a provider. One team may retain full prompts in logs, while another cannot reconstruct an incident at all. A change to a safety rule can become an application release across multiple repositories.
The practical alternative is one governed doorway between applications and model providers. It centralises policy enforcement while preserving provider flexibility and keeping business logic unchanged.
The security controls a production LLM needs
A useful LLM security design works at request level. It should inspect what enters the system, control what data can leave it, select an approved execution path, assess what comes back, and record enough evidence to investigate decisions without creating a second sensitive-data store.
Prompt injection and jailbreak shielding
Prompt injection is not limited to users typing hostile instructions into a chat box. It can arrive inside uploaded documents, web content, retrieved knowledge-base passages or data returned by another system. The model may be asked to treat untrusted text as authority, disclose hidden instructions or bypass a workflow’s intended constraints.
A prompt shield should evaluate requests before routing. It cannot make a model immune to manipulation, and it should not be presented as a guarantee. It does, however, give teams a consistent enforcement point for known injection and jailbreak patterns across providers and applications.
The operational question is what happens when a request is suspicious. Security teams may need a block for high-risk workflows, while a lower-risk internal assistant might require a warning and trace for review. The policy should match the workflow’s consequence, not a generic notion of safety.
DLP for sensitive inputs
Sensitive data is often legitimate business context. A customer service assistant may need an account number; an HR workflow may encounter personal information. The goal is not to prohibit all such use automatically. It is to apply a defined data policy before information is sent to a model.
DLP controls can log, warn, redact or block matches. Redaction replaces the detected value with [REDACTED], which reduces disclosure to downstream providers but also changes the context available to the model. That trade-off matters. For a request where exact personal details are unnecessary, redaction is usually appropriate. Where the value is essential to an authorised process, the correct decision may be to allow it under a documented policy rather than create an unreliable workaround.
Treat DLP as a data-minimisation control, not as a way to reconstruct sensitive values later. The safer architecture is to avoid exposing values that the model does not need in the first place.
Output moderation
Security does not end when the provider accepts the prompt. Model output can include harmful advice, sensitive material, policy-violating language or an answer that is unsuitable for the channel in which it will appear.
Output moderation creates a final decision point before a response reaches a user or downstream system. Its policy should be specific to the product. An internal research tool and a public customer assistant have different tolerance for content, different audiences and different escalation paths.
The order of these controls matters. A controlled request path should run prompt shielding first, then DLP, then routing, followed by output moderation. That sequence prevents an unsafe prompt from proceeding, protects sensitive input before provider selection, and assesses the generated result before release.
Routing is part of the security architecture
Routing is often categorised as a cost or resilience feature. It is also a governance control. Without a central routing layer, applications may make provider and model decisions independently, making it difficult to enforce approved providers, regional requirements or workload-specific priorities.
Named routes and routing rules let platform teams define an execution policy once, rather than embed it in every service. A route can select the most cost-effective capable model according to defined priorities and use automatic failover when a provider is unavailable. The application continues to call a stable endpoint, while the platform team retains control over the model estate.
This does not mean every request should go to the cheapest model. High-stakes extraction, legal review support and customer-facing decisions may justify a different capability threshold from routine classification or summarisation. Good routing makes those choices explicit and measurable.
Logging that supports investigation without excess retention
LLM observability can create its own data-protection problem. Full prompt and response retention may help debugging, but it also duplicates customer, employee or proprietary data into another system. Conversely, sparse logs can leave teams unable to explain a blocked request, a provider failure or a sudden cost increase.
Metadata-only logging by default is a sensible baseline. Per-request traces can record the operational facts teams need: route and provider decisions, policy outcomes, latency, usage and cost signals. Payload retention should be an explicit opt-in decision with a clear purpose and appropriate handling.
This distinction is particularly relevant for regulated organisations. Audit trails should show that policies were applied and decisions were traceable, without assuming that retaining every raw conversation is necessary. Evidence should be proportionate to the risk and the retention obligation.
Spend limits are a security control too
Unbounded LLM spend is an availability and governance issue, not merely a finance problem. A looping workflow, abused endpoint or unexpected usage surge can consume budget rapidly. If that happens unchecked, the impact can spread beyond the individual application.
Hard spend caps and budgets turn a financial policy into a request-level control. When a cap is reached, the request can be blocked with a 429 response rather than silently exceeding an agreed limit. Teams should decide in advance which services can fail closed, which require an operational escalation path, and how close-to-cap alerts are handled.
The trade-off is deliberate: strict caps protect budgets but may interrupt a legitimate peak in demand. That is preferable to discovering the issue after uncontrolled usage has accumulated, provided product owners understand the expected behaviour.
Implement one governed doorway without rewriting applications
Security programmes stall when the integration requires every engineering team to adopt a new SDK or redesign its business logic. An OpenAI-compatible gateway reduces that friction. The application can retain its existing client pattern while sending requests through the control plane.
import os from openai import OpenAI
client = OpenAI( api_key=os.environ["ROUTEUR_KEY"], base_url="https://api.routeur.ai/v1" )
response = client.chat.completions.create( model="auto", messages=[ {"role": "user", "content": "Summarise this support case."} ] )With this approach, the application change is limited to the key and base URL. Its business logic stays byte-for-byte identical, while routing, prompt shielding, DLP, output moderation, budgets and traces operate consistently behind the endpoint.
routeur.ai is designed for this model of deployment: one control plane for multi-provider LLM use, with EU data residency in GCP europe-west1, metadata-only logging by default and audit trails that support EU AI Act record-keeping. For a platform team, the value is not another dashboard. It is the ability to apply the same operating rules wherever LLM requests originate.
Make policies testable before they become incidents
A policy is only useful if teams can validate its effect. Start with representative requests from actual workflows: a normal support query, a prompt injection attempt embedded in retrieved text, an input containing personal data, and an output that should be moderated. Review the trace for each case and confirm the expected action.
Use dry-run testing where an enforcement decision needs review before it affects production traffic. Then phase controls in according to consequence. Logging or warnings may be appropriate while teams tune detection patterns; redaction or blocking may be required once the risk and workflow are understood.
The objective is not to claim that an LLM-powered system is perfectly safe. It is to ensure that each request passes through controls your organisation can inspect, adjust and evidence. When models, providers and product teams change, that governed doorway should remain the place where policy holds.