A support copilot receives a customer’s message containing their name, phone number, account number and delivery address. If the application forwards that payload unchanged to an external model provider, the exposure has already occurred. To mask PII before sending to LLM services, the control must sit on the request path, before provider routing, prompt logging or model inference.
For production AI teams, this is not a prompt-writing preference. It is a data-flow control. The question is whether every outbound request is inspected, transformed according to policy and recorded with evidence that sensitive values did not leave the governed boundary.
Why mask PII before sending to LLMs
LLM applications tend to collect data from places built for human workflows: CRM records, support tickets, call transcripts, HR documents, invoices and free-text forms. That data is useful context, but it often contains personally identifiable information (PII), payment details, credentials or identifiers protected by contractual and regulatory obligations.
A provider's enterprise terms, retention settings and regional processing options can reduce risk. They do not remove the need to minimise what leaves your environment. A provider policy may say prompts are not used for training, yet sending a customer's unneeded email address or national insurance number remains an avoidable disclosure.
Masking also reduces the blast radius of operational tooling. Teams frequently enable request logs to diagnose model quality, monitor token costs or investigate failures. If raw payloads are permitted through the gateway, those logs can become a second sensitive-data store. A sound design applies the same control before routing and ensures observability retains only metadata or an approved, sanitised representation of the request.
Masking is not the same as deleting context
The right transformation depends on what the model needs to do. Blind redaction can degrade results. A model asked to compare two support cases may need to know that the same customer appears in both, but does not need the customer's actual name.
Tokenisation preserves that relationship. A gateway can replace sarah.jones@example.com with [EMAIL_4f92] and replace every matching occurrence in the request with the same token. The model can then reason about repetition, ownership and sequence without receiving the underlying value. The application can restore authorised values only after the response returns, and only where restoration is required.
Redaction is the better choice when the value carries no semantic value. For example, replacing a full card number with [PAYMENT_CARD_REDACTED] prevents disclosure without pretending that the model should use it. Generalisation can be useful for analytics tasks: an exact date of birth might become an age band, and a postcode might become a region.
These choices should be explicit policies, not ad hoc regular expressions embedded across application services. A legal-review assistant, for example, may need named entities within a privileged document and therefore require a tightly scoped internal model route. A customer-service classifier usually does not. One global rule is rarely sufficient.
Put the control at the governed doorway
The most dependable architecture places DLP inspection in an API gateway between applications and model providers. Every OpenAI-compatible request enters one governed doorway. The gateway identifies the calling team, application, environment and policy through API credentials and custom headers, scans the payload, applies the appropriate transformation, then sends the minimised request to the selected provider.
This placement matters because it covers direct product calls, background jobs, internal copilots and new services without relying on every development team to implement identical safeguards. It also means policy can be changed centrally. No application rewrite is needed when a security team adds a new identifier type or blocks a newly discovered pattern.
A practical request path looks like this:
Application request -> identity, environment and policy selection -> PII detection and masking -> prompt-injection inspection -> model routing and provider call -> output moderation and authorised token restoration -> metadata-only audit eventThe order is intentional. Sensitive values should be masked before the payload reaches downstream inspection systems that do not need them. Output moderation should run before restoration, so an unsafe output is never enriched with original values. Routing then chooses a capable, cost-effective approved model without changing the protection applied to the request.
Build policies around data classes and purpose
Start with a small data classification model that maps to real application use cases. Most teams need separate treatment for direct identifiers such as names, email addresses, telephone numbers and addresses; government and financial identifiers; authentication secrets; and sensitive business information such as contract terms or source code.
Detection should combine structured recognisers, pattern matching and contextual classifiers. Regex is appropriate for formats with strong structure, such as card numbers after validation checks. It is weak on names and addresses, where context determines whether a phrase identifies a person. Classifiers improve coverage but can create false positives, so policies need confidence thresholds and clear handling for uncertain matches.
Do not treat detection accuracy as a reason to allow raw data by default. Use a fail-closed rule for high-risk classes. If a request appears to contain a credential, payment card or government identifier, block it and return a machine-readable policy response to the application. For lower-risk direct identifiers, tokenise or redact according to the route and task.
The policy should also account for data residency and model eligibility. A masked request may be suitable for a lower-cost external model, while an unmasked request approved for a narrow workflow may require a specific private deployment. Routing priority and DLP policy are related controls, but they should remain independently auditable.
Treat re-identification as a privileged operation
Masking fails if the lookup table becomes another uncontrolled data store. Keep the token-to-value mapping separate from provider-facing payloads, encrypt it, restrict access and define a short retention period. The mapping should be bound to a request or session context so that a token from one tenant cannot be resolved in another.
Restoration should happen only when an authorised downstream system needs the original value. A human-facing support draft may need the customer's first name inserted before it is shown to an authenticated agent. A classification score, quality evaluation or analytics event does not. In those cases, keep tokens in place or discard them.
This is also where output handling deserves attention. Models can reproduce strings from their input, infer sensitive details from context or follow malicious instructions that ask them to expose hidden data. Scan responses for policy violations before restoration, and prevent output from containing tokens that were never present in the current authorised context.
Keep evidence without retaining payloads
Compliance and incident response require proof that controls ran. They do not require a warehouse of raw prompts. Store metadata such as request ID, tenant ID, model route, policy version, detector categories, masking count, block decision, latency, token usage and retention status. That record can show which policy was enforced and where a request was processed without retaining the customer's content.
This distinction makes investigation practical. A security team can identify a surge in blocked credential submissions from a particular application, while FinOps can analyse routing and spend by model. Neither team needs unrestricted access to raw conversations to do its job.
Define an exception process as well. Security teams will encounter legitimate workflows that need more context than the default policy permits. Exceptions should be scoped to a named application, data class, provider route and expiry date, with an accountable owner. A permanent bypass header is not an exception process. It is a future incident.
Deploy without creating a second integration project
The fastest adoption path is to preserve the application contract. With an OpenAI-compatible gateway, teams typically change the base URL and API key, then attach a policy or environment header where needed. Your business logic stays byte-for-byte identical while request-level controls are applied centrally.
client = OpenAI( base_url="https://gateway.example/v1", api_key=os.environ["GATEWAY_API_KEY"] )
response = client.chat.completions.create( model="routing-default", messages=messages, extra_headers={"X-Policy": "support-production"} )Before broad rollout, run representative traffic in observe mode. Measure detected classes, false positives, masking latency and model-quality changes. Then enforce masking for low-risk workflows first, followed by blocking rules for secrets and regulated identifiers. Routeur.ai is designed for this model: policy enforcement, model routing, controlled logging and spend controls operate at the same gateway layer rather than through separate middleware services.
A useful operating target is not simply “zero PII sent”. It is “only the minimum authorised context leaves each boundary, with evidence for every request”. That standard gives product teams room to build capable AI features while giving security and compliance teams a control they can inspect, tune and trust.