The question arrives eleven months later and it is never phrased as an accusation. Someone in a review, or a consultant doing operational due diligence, or a client who has read something in the press, asks why this particular name was on the desk's shortlist on a particular Tuesday. The honest answer at most shops is that the scanner surfaced it and an analyst liked it. That is not an answer. It is a description of an event with no evidence attached, and the person asking will correctly hear it as such.
What makes this harder than the equivalent problem for a rules-based screen is that a language-model scanner is not a formula you can print. You cannot hand over a spreadsheet with the filter in cell B2. The output is a function of inputs you did not all specify, some of which live at a vendor and change without telling you. So the record has to be built deliberately, at scan time, or it does not exist.
The standard is reconstruction, not explanation
Set the bar in the right place before you design the record. The bar is not "can we explain in words why this name appeared". Words are cheap and post hoc explanations are worse than useless because they sound authoritative.
The bar is reconstruction. Given the record alone, without asking anyone what they remember, can a person who was not there determine what was asked, what set of instruments could have answered, what produced the answer, and what evidence each line rested on. Full bit-for-bit reproduction of the output is a nice-to-have and often impossible. Reconstruction of the conditions is achievable and is what a reviewer actually wants.
The practical difference shows up when a name goes wrong. Under an explanation standard, the desk writes a paragraph and everyone nods. Under a reconstruction standard, you can show that the universe on the day excluded a peer that later moved, and that the exclusion was a documented mandate constraint rather than an oversight. One of those closes the question. The other one keeps it open.
The fields a scan record has to carry
Six fields, captured automatically at run time. Anything that depends on someone remembering to fill it in will be empty within a month.
- A hash of the request. Not a description of it. A hash of the exact text and parameter set, stored alongside the text itself. The hash is what lets you group scans that are genuinely comparable and separate the ones that are not, and it is the single field that makes everything downstream analysable rather than merely archived.
- A universe snapshot. The identifiers of everything eligible to appear, resolved at scan time, not the description of the universe. "Liquid US mid caps" is not a universe, it is a nickname for one, and the set it resolves to changes weekly. Store the list, or store the exact query plus the as-of date of the reference data it ran against.
- Model identity and version. Covered below, because this is the field that gets skipped.
- The run timestamp with timezone and seconds. Sentinel gets this right in the header, stamping the last scan to the second with an explicit zone. Copy that convention. A desk that spans time zones and logs dates without zones will eventually have two records that cannot be ordered, and the one place that always matters is the boundary either side of a market event.
- Source identifiers per surfaced line. For each name, what specific artifact the claim rests on. A filing accession number, a headline identifier, a price series with its as-of. Not a URL alone, because URLs rot, and rot is silent.
- The control state of the tool. Filters applied, and the run configuration. In Sentinel that means the direction, urgency, asset class and market status selections, plus whether the scan was a manual run or an automated one and whether auto-run was even enabled. The screenshot shows why the last part is not pedantry. A kill switch active and auto-run off produces a result set that looks completely normal and is not current.
Model version is the field everyone forgets
The reason this one gets skipped is that it does not feel like your data. It feels like plumbing at a vendor. But it is the input with the largest effect on output and the shortest half-life.
Model endpoints get updated. Providers retire versions, sometimes with notice and sometimes with a silent swap behind a stable alias. A desk that records "we used the scanner" has recorded nothing, because the scanner in March and the scanner in September may be different systems wearing the same name. When your hit rate changes across a quarter, the first question a serious reviewer asks is whether the model changed, and if you cannot answer it you cannot attribute the change to your process at all.
This also matters for anything multi-stage. Where a pipeline uses different models for different steps, each step needs its own version stamp, because a change to a scoring step and a change to a mapping step have entirely different signatures in the output. Ask your vendor for the version string per call and store it. If a vendor cannot tell you which model version served a given call, that is a genuine finding about the vendor, and it belongs in your due diligence file rather than being quietly absorbed.
Write down what you cannot reconstruct
An audit record that implies more than it can deliver is a liability. Three limits are worth stating explicitly in the procedure document rather than discovering under questioning.
The first is nondeterminism. Language model output varies between runs even with identical inputs, so replaying a scan may not reproduce the same list. Say so. The record establishes what was asked and what came back, not that the same question would come back the same way. A reviewer who understands the technology will accept that immediately, and will be considerably less impressed by a shop that claims otherwise.
The second is vendor retention. Your logs may outlive the provider's ability to serve the model that produced them. Retention on your side is a choice you control. Reproducibility on the vendor side is not.
The third is source decay. A headline identifier resolves to a story that may be edited or withdrawn. If the source matters to a position, snapshot the content, not just the pointer.
Ownership, retention, and the sampling routine
Records nobody reads decay into records nobody can read. Assign an owner, usually whoever owns the research systems rather than whoever runs the scans, and give them two standing jobs.
The first is a retention policy that matches your longest plausible lookback, which is typically driven by your regulator and your client contracts rather than by your own curiosity. Store the scan records with the same seriousness as trade records, because in an inquiry about a specific position they will be requested together.
The second is a monthly sample. Pull three scans at random, hand them to someone who was not involved, and ask them to reconstruct why one named line appeared. Time how long it takes. This is the only test that actually works, because every audit record looks complete when you design it and the gaps only appear when a stranger tries to use it. When the reconstruction fails, the failure is almost never the missing exotic field. It is that the universe was stored as a phrase, or the timestamp was stored without a zone, or nobody thought the model version was worth a column.