To audit AI prompts and outputs in a regulated enterprise, you log three things at request granularity: the exact prompt sent to the model, the exact output returned, and the version of the model, system prompt and retrieval index that produced it. Most enterprise deployments log none of these. They monitor the model, its uptime, its token spend, its latency, and call that governance. When a regulator asks what the system told a specific person on a specific date, the honest answer is that nobody kept the receipt.
That is the governance gap. It sits in the record, not in the model.
Monitoring the model is the easy 20 percent
Every enterprise AI dashboard tracks the same things: requests per second, error rate, average latency, cost per thousand tokens, GPU utilisation. This is operational telemetry borrowed straight from the standard web-service playbook, and it is genuinely useful. It tells you the service is up. It tells you the bill is not about to triple. It tells you nothing about what the system actually said.
In a regulated context the interesting failures do not show up in any of those metrics. A model that returns a confident, fluent, wrong answer has perfect latency and a clean error rate. A prompt that quietly started pulling the wrong patient record still resolves in 300 milliseconds. A system prompt edited by an engineer on a Friday afternoon that changed how the model handles a consent question produces no alert, because from the infrastructure's point of view nothing broke. The dashboard is green. The liability is not.
The reason enterprises stop at model monitoring is that it is the part the platform gives you for free. Prompt and output logging you have to build, and building it forces uncomfortable decisions about storage, retention, and who is allowed to read a log that now contains sensitive input.
What prompt-level audit actually means
Prompt-level audit is the ability to reconstruct any single interaction after the fact. Not a sample. Not an aggregate. The specific one a regulator, a lawyer, or an angry patient is asking about.
For each request that means capturing four things and tying them together with one identifier:
The full prompt as the model received it, including the system prompt, the retrieved context injected by your retrieval layer, and the user input. In a retrieval-augmented system the retrieved chunks are where most of the risk lives, and they are the first thing teams forget to log because they are generated at runtime and never written down anywhere.
The full output as the model produced it, before any downstream formatting stripped or reshaped it. If your application post-processes the model's answer, log both the raw response and the version the user saw.
The versions in force at that moment: model identifier and revision, system prompt hash, retrieval index snapshot, and any tool or function definitions available to the model. Without this you can reproduce the text but not the conditions, and in a drift investigation the conditions are the whole question.
The human decision, where one exists. If a clinician, an agent, or a reviewer accepted, edited, or overrode the output, that action belongs in the same record. This is the human override capture that turns a log from a transcript into an accountability trail.
NIST's Generative AI Profile makes the point in governance language: teams routinely underestimate the need for traceability until a bad response has already gone out, at which point the record either exists or it does not (Source: NIST AI 600-1 Generative AI Profile, NIST, July 2024). You cannot retrofit a log onto a conversation that already happened.
Drift is the failure model monitoring cannot see
Drift in a regulated deployment rarely looks like the textbook data-drift charts. It is quieter and more human.
A prompt-level regression: someone tunes the system prompt to fix one edge case and shifts behaviour on a class of inputs nobody re-tested. A retrieval-index change: the knowledge base gets re-indexed, the chunking changes, and answers that used to cite the current SOP now cite last year's. A silent model update on a hosted endpoint, where the provider ships a new revision behind the same name and your outputs change without a single line of your code changing. None of these move latency or error rate. All of them can move what the system tells a patient about a drug interaction or what it tells a compliance officer about a filing deadline.
You catch this only by comparing outputs across time, which requires that you logged the outputs and pinned them to versions in the first place. Drift detection is not a separate product you buy. It is a query you can only run if the record exists. This is why observability has to be a design decision at deployment, not an add-on after the audit request lands.
Why this is worse in regulated industries
In an unregulated setting a wrong output is a bad experience. In a hospital or a pharma enterprise it is a documented event with a liability owner, and under DPDP 2023 it is often a processing action on personal data that the Data Fiduciary is answerable for.
Liability in these systems does not sit in one place. Clinical judgment stays with the clinician and the hospital. System behaviour, an output landing outside its intended-use envelope, a note attributed to the wrong encounter, retrieval pulling the wrong record, sits with the vendor. Deployment, a site running the tool outside its stated use or a reviewer signing an output unread, sits with the operator. That allocation is the sane way to divide responsibility. It is also completely unenforceable unless you can reconstruct what happened. Every one of those categories is a factual claim about a specific past interaction. No log, no facts. No facts, and the liability defaults to whoever has the weakest lawyer.
This is the observability position Nextdot ships as standard on every deployment: audit logs, versioned models and prompts, human override capture. Not because it is a nice governance gesture, but because the contract's liability split is fiction without it. The uncomfortable market observation still holds: walk into a hospital running an incumbent AI vendor and ask to see the last thirty times the system was overridden, and there is almost never an answer. The model was monitored. The prompts were not.
What good looks like, concretely
A deployment that can survive a regulatory question has a few properties you can check for before signing anything.
Every request is reconstructable end to end from a single trace identifier, prompt through retrieved context through output through human action. Model, prompt, and index versions are pinned to each request, so a change in behaviour can be traced to a change in configuration. Logs are retained for a period that matches the regulatory limitation window for the domain, not the default 30 days a logging platform ships with. Access to the logs is itself controlled and audited, because a prompt log in a clinical system now holds sensitive input and cannot sit in a channel any engineer can read. And overrides are queryable, so the last thirty are a report, not an archaeology project.
None of this requires exotic tooling. It requires deciding, at build time, that the record matters as much as the response. Most teams decide the opposite by default, because the model is the exciting part and the log is the boring part. In a regulated deployment the boring part is the one that holds up in front of the Data Protection Board of India.
Frequently asked questions
What is prompt-level audit?
Prompt-level audit is the ability to reconstruct any single AI interaction after the fact: the exact prompt the model received including system prompt and retrieved context, the exact output it produced, the model and configuration versions in force at that moment, and any human decision to accept or override the result, all tied together by one trace identifier. It is the difference between knowing your AI service was running and knowing what it actually told a specific person on a specific date.
Why is monitoring the model not enough?
Model monitoring tracks uptime, latency, error rate, and cost. A model that returns a confident wrong answer, or one whose behaviour shifted after a prompt edit or a silent provider update, scores perfectly on all of those. The failures that matter in a regulated setting do not register as infrastructure errors. You only see them by inspecting the prompts and outputs themselves, which model monitoring does not capture.
How do you log AI outputs for compliance?
Capture the raw output as the model produced it, before any downstream formatting, and store it against a trace identifier that also links the full input prompt, the retrieved context, the model and prompt versions, and any human review action. Retain the record for the regulatory limitation window relevant to your domain rather than a default retention period, and control and audit access to the logs, since they now contain sensitive input. The test is simple: can you produce the exact interaction a regulator names, with the conditions that produced it.
What is prompt drift?
Prompt drift is a change in what an AI system outputs that comes from a change in its configuration rather than its code: a tuned system prompt that shifts behaviour on inputs nobody re-tested, a re-indexed knowledge base that changes which source answers cite, or a hosted model silently updated behind the same name. It does not move latency or error rates, so operational monitoring misses it entirely. You detect it only by comparing logged outputs across time against pinned versions, which requires that you logged and versioned them from the start.