A regulator, customer or internal risk committee asks a deceptively simple question: why did this AI system produce that result on that day? Useful EU AI Act records let a team answer without reconstructing an incident from scattered application logs, provider consoles and policy documents.
That does not mean retaining every prompt and response indefinitely. For LLM systems, indiscriminate payload storage can create a second problem: a growing repository of personal, confidential or commercially sensitive information. The operational challenge is to preserve evidence of decisions and controls while applying data minimisation deliberately.
What EU AI Act records actually mean
Record-keeping duties under the EU AI Act are not one universal logging rule for every organisation using AI. The required evidence depends on the organisation's role, the system's classification, its intended purpose and the obligations that apply at that point in the value chain.
For high-risk AI systems, automatic logging is a core traceability requirement. Providers must design systems so that events can be recorded over their lifetime, while deployers must retain automatically generated logs under their control for an appropriate period, generally at least six months unless another applicable law requires otherwise. Technical documentation, quality-management evidence, post-market monitoring and incident handling may also be relevant, but they are not substitutes for event records.
A general-purpose AI model introduces a different set of responsibilities, particularly for its provider. An organisation deploying an LLM application still needs to determine what it is actually doing with the model. A customer-support assistant is not automatically high-risk merely because it uses a powerful model. Its intended purpose, sector and effect on people matter. The same model used to support a decision in employment, education, credit or access to essential services may demand a far more stringent assessment.
This classification work should happen before a retention setting is selected. Legal, compliance and product teams need an agreed record of the intended purpose, system boundaries, human role, affected users and risk controls. Engineering then needs to make that record testable in production.
Build EU AI Act records around evidence, not volume
The useful question is not, “How many logs do we have?” It is, “Can we demonstrate what happened, what controls were active and who was accountable?” That shifts record design away from raw telemetry towards an evidence model.
For an LLM application, a request-level record will normally need to establish the time of the event, the application or service involved, the logical route or model selected, the provider used, the policy outcome and a trace identifier that connects related events. Where relevant, it should also show whether a request was blocked, redacted, moderated or completed, alongside the configuration version that governed that outcome.
This is more useful than a line saying only that an API call succeeded. A successful response may still require investigation if a policy was changed shortly before the request, if a fallback provider was selected, or if an output-moderation rule intervened. The record should make those facts observable without treating sensitive prompt content as the default audit artefact.
Keep the control plane in scope
Many organisations capture application logs but omit the configuration that shaped a decision. That creates an avoidable gap. A trace can show that a request was sent to a model; it cannot, by itself, prove which routing priority, DLP action, budget rule or moderation policy applied at that moment.
Maintain change records for the controls around the model as well as the request. That includes approved policy changes, named-route changes, model availability decisions, access changes for scoped API keys, and the identity of the person or process that approved a material alteration. Versioned configuration and change approval provide the context that turns an event log into defensible evidence.
There is a practical boundary here. Not every tuning adjustment needs the same workflow. A material change to the intended purpose, the system's risk profile or the action taken on sensitive data warrants stronger governance than a routine operational adjustment. Define that distinction in advance rather than debating it during an incident.
Capture human accountability where it exists
Records should also reflect where people make or review consequential decisions. If an AI output is presented to an employee for review, record the review process, escalation path and authority to override. If the application acts automatically, document the circumstances in which that is permitted and the monitoring that detects unacceptable behaviour.
This is especially relevant where a team describes a person as “in the loop” but cannot show what that person saw, what they could change or when they intervened. Human oversight is an operational control, not a label.
Record the request lifecycle in the right order
For production LLM systems, evidence should mirror the sequence in which controls are actually applied. Otherwise, an audit trail can suggest a control acted on data it never received.
A governed request lifecycle begins with prompt shielding, which evaluates prompt-injection and jailbreak attempts. DLP then applies its selected action to sensitive information: log, warn, redact with [REDACTED], or block. Only then is the request routed according to the configured rules and priorities. Output moderation evaluates the response after it is generated.
Each stage can produce meaningful evidence. Prompt shielding can show that a request was stopped before reaching a provider. DLP can show that a redaction or block occurred. Routing can identify the decision path and selected provider or model. Output moderation can show that a generated response was subject to a safety control before it reached the user.
Do not overstate what these records prove. A moderation outcome records the behaviour of a configured control at a point in time; it does not establish that every harmful output is impossible. Equally, a clean trace does not eliminate the need for monitoring, user reporting and incident investigation.
Retention must balance traceability and minimisation
Prompt and response retention is often the hardest design decision. Full payloads can help investigate a disputed result, reproduce a defect or identify a missed policy condition. They can also contain personal data, client secrets, source code and regulated information that should not be retained simply because storage is inexpensive.
A proportionate model separates metadata from payloads. Metadata-only logging can retain the timing, route, policy outcomes, identifiers and usage facts needed for oversight, while payload retention is an explicit, narrowly controlled choice for use cases that justify it. Access to any retained payloads should be limited, logged and reviewed, with a defined deletion schedule.
The retention period should be based on the applicable obligation, the system's risk, the likely time to detect an issue and the needs of investigations or dispute handling. “Keep everything forever” is not a serious retention policy. Nor is a short default period that makes a high-risk incident impossible to investigate. Teams should document the rationale, test deletion in practice and review it when the system or its risk profile changes.
Data protection obligations remain relevant throughout. EU AI Act record-keeping does not remove the need for a lawful basis, purpose limitation, security measures, data-subject rights processes and processor oversight where personal data is involved.
Make records usable during an audit or incident
An evidence store is only valuable if the right people can retrieve and interpret it under pressure. Test it with a realistic question: a user reports a harmful or inaccurate output from a particular date. Can the team identify the application, trace the request, establish the active control configuration, determine whether a policy action occurred and show who owned remediation?
This test usually exposes fragmentation. Application teams may hold request IDs, security may hold policy changes, FinOps may hold spend controls and procurement may hold provider commitments. One governed doorway between applications and model providers reduces that fragmentation by creating a consistent decision point. routeur.ai, for example, provides per-request traces and metadata-only logging by default, with payload retention as an explicit opt-in choice.
The implementation objective is not a larger logging estate. It is a smaller number of reliable records that connect technical behaviour to accountable controls. Start with one production workflow, define the evidence needed for its risk profile, rehearse retrieval, then extend the pattern across routes and teams. When a serious question arrives, the quality of the answer will depend less on how much data was stored than on whether the records explain the decision.