Why we store our own run record
Several of the things published in May 2026 come back to the same question: what a system records, and who controls whether it keeps recording it. The most direct of them is a vendor notice withdrawing a set of audit values that people have been relying on. So this month we have written out what we expect a run record to hold, field by field, and why we do not count on the platform to hold it for us.
The questions that follow a system going live tend to be the same kind of question. A supplier rings about an invoice, an auditor asks about a payment, a manager notices a record that looks wrong. Someone has to reconstruct what one run did on one day. A run record that holds enough detail makes that a few minutes of reading. Where the detail is missing, someone has to redo the process by hand to work out what the run did, and that is usually the same person who is already on the phone to the supplier.
The platform log is the obvious place to look for this, so it is worth being plain about what that log is for. Platform logs exist to help you run the platform. A platform log will show that a job started, which worker picked it up, how long it ran, and whether it ended in an exception. Nothing in it identifies the document the run read, the fields the model extracted, the validation rule that rejected a value, or the person who approved the payment that went out. A platform log is built to answer operational questions about the job, and the questions that arrive after go-live are about the contents of one run.
The other reason we keep our own is that vendors change what their platforms log, and you do not get a vote. Microsoft published a Power Platform breaking-change notice dated 22 May 2026 stating that Dataverse will stop including before-and-after field change values in the audit events sent to Microsoft Purview, starting in May 2026. Anyone depending on those values has to find another source for them. We would rather write the run record ourselves into a database we control than discover in a release note that a field we relied on has gone.
Inputs, kept in the form they arrived
Every run starts with something arriving: a file in a folder, an email, a webhook payload, a row returned by a query. We store that input as received, with enough metadata to identify it later. For an email that means the message id, sender, subject, received time, and the attachment bytes or a pointer to them in blob storage. For an API trigger it is the raw JSON body and the headers the process depends on.
Keeping the raw form is what makes a run reproducible. Six weeks later, when someone disputes an output, you can feed the same bytes through the current pipeline and see whether it behaves differently. That test needs the original bytes, because a stored parsed version was produced by the parser and cannot be used to check it. This is the field we see dropped most often on storage grounds, and we would argue hard for keeping it.
Some steps produce no request and response pair to store, and that class of step is becoming more common. Microsoft Learn records that computer use reached general availability in Copilot Studio in May 2026, letting agents automate web and desktop apps by controlling browsers and desktop applications on behalf of users. When a step drives a user interface, we want the record to hold screenshots, the target application and window, and the values read off the screen.
Decisions, model outputs and confidence
A decision is any point where the run could have gone more than one way. We store the decision point's name, the values it looked at, the branch taken, and the rule or model version behind it. Deterministic decisions in Python are cheap to record, because the rule lives in versioned code and we keep its id and the values compared.
Where a language model does the reading, classifying or interpreting, we store more: the model identifier, the prompt template version, a reference to the input we sent, the raw response text, the parsed structured output, and the per-field confidence. We keep the raw response beside the parsed one so that a bad extraction can be told apart from a bad parse when someone reviews the run.
We store the eventual outcome beside each confidence value. Over enough runs, the pairs of confidence value and eventual correction show where to set the threshold for each field. We do not put weight on a reported confidence number on its own until it has been paired with outcomes on the client's own documents. Mozilla engineers writing on 7 May 2026 about their Firefox security work said their earlier experiments with language models showed some promise, but that a high rate of false positives made them impractical to scale. Measuring a false-positive rate on our own work needs both the model's claim and the recorded outcome, so both fields sit in the run record.
Validation results, including the checks that passed
Validation in our builds is Python. The rules cover mandatory fields per record type, format checks on each identifier, cross-field consistency, date sanity, and a confidence floor per field. Each check writes a row to the run record: check id, field, the value tested, pass or fail, and the message.
We store the passing checks as well as the failures, which costs storage and gets queried on most builds. When a bad record gets through, the first question we ask is whether the relevant check ran and passed or never ran at all, and answering it requires the passing rows to be present in the record.
Policy enforcement produces decisions too, and they belong in the same record. UiPath's May 2026 cloud platform release notes record that centralised guardrails entered public preview in Automation Ops, letting administrators enforce safety policies across agents and automations, with guardrail types for harmful content, IP protection, PII detection and prompt injection. We record which policy evaluated a step and what it returned, whichever enforcement layer produced the result.
Approvals and the person who gave them
Outbound work in our builds is held for a person to approve by default. The approval row names the person, the time, the decision, and any edits they made before approving. We record a named user in that field, because a service account id does not identify anyone.
The payload the approver was shown is the field most often left out, and it is the first thing we go looking for in a system we did not build. An entry reading "approved by J Smith at 14:32" is worth little if the content has been regenerated since, so we store the rendered payload with the approval. If you inherit a system and that field is absent, treat every approval in the history as unevidenced, because there is no way to show afterwards what the person agreed to.
Two things published in May bear on this. Simon Willison, writing on 5 May 2026 about an Andon Labs experiment running a Stockholm cafe with an AI manager, argued that experiments like it need to keep their own human operators in the loop for outbound actions that affect other people. Microsoft Learn lists, under May 2026, a preview capability that automatically creates Microsoft Entra Agent IDs for each agent, so that connector permissions, Conditional Access policies and DLP governance can be scoped to individual agents.
Outputs written, errors, and the record itself
For every write, we store the target system, the endpoint or table, the payload sent, the response, and the identifier the target returned. That returned id lets you go the other way later, from a row in the finance system back to the run that made it.
For an error, we store the exception type, the message, the stack trace, the step that raised it, the input being processed, the retry attempt number, and the final state of the run. We also record what had already been written before the failure, because unwinding a partial write requires knowing exactly what landed.
The run record is a data holding in its own right, often the most sensitive one, because it contains copies of documents and extracted personal details. The Office of the Australian Information Commissioner released its Australian Community Attitudes to Privacy Survey on 28 May 2026, reporting that 87% of respondents are more concerned about privacy than they were five years ago, and that just 4% believe AI companies are worthy of their trust. We do not read that 4% as a verdict on any one system, since a question about AI companies in general is a different question from one about the tool a person's own employer runs, though it is the climate any record store now sits in. Retention periods, field-level redaction, and access control on the record store are part of building it, and we set them with the process owner.
Retention length is the one part of this we do not think anybody has settled. An auditor may want the record years after the run, and for that whole time it holds document copies and personal details, so no single default period is defensible in both directions at once. We have yet to see a house standard that survives contact with a second process. What we do is set the period per process with the owner, put it in writing, and review it on a schedule.