A DFIR agent that can only report what it can prove a tool said.
A finding carries the record it came from, the field, and the value it claims that field holds. Before anything reaches the report, plain Python re-opens the captured tool output and compares. No second language model is asked, because a model can be argued out of a correct refutation and a comparison cannot.
The clone runs the proof — fixtures, the adversarial catalogue, the whole case in three seconds. The install points the gate at your evidence. The package is witness-cli because witness on PyPI was taken in 2017; the module and the command are still witness.
Nobody worries about the agent that invents a tool it never ran — that one is caught the moment somebody looks. The dangerous finding is the one where the tool genuinely ran, the parent process really is PowerShell, the story is exactly right, and two digits changed places on the way to the sentence.
Hand that to a second language model and ask it to check. It reads a fluent, internally consistent account of a real intrusion and agrees, because the claim is overwhelmingly plausible — and plausibility is the only thing it can measure. The transposition is not a reasoning failure. It is a transcription failure, and it needs a reader, not a critic.
Both gates below run on real fixtures from the SANS 2018 compromised image. Nothing here is a mock-up of the logic — it is the logic, which is small enough to fit on a page precisely because it is not a model.
Neither gate has an opinion, a temperature, or a bad day. Gate 1 is a comparison against a record the finding itself named. Gate 2 is a lookup for a tool that either ran or did not. That is the entire oracle — and it is why the same case, run twice, produces the same report to the byte.
Checking a language model with a language model is the reflex, and it does catch things. But it inherits the property that made the first model unreliable: it is a persuadable reader of prose, being asked to referee a claim about a number.
A transposed PID inside an otherwise flawless account of a real intrusion reads as correct, because it is correct in every respect the judge can assess.
It can be talked round. A confident restatement of the claim moves the verdict, which is exactly the failure being defended against, one layer up.
It is non-deterministic, so the same report checked twice can come back differently, and neither answer can be audited later.
It reads one field of one named record and compares. It has no view on whether the story is compelling, which is why the story cannot sway it.
It cannot be argued with, because there is no argument to make — the record either holds that value or it does not.
It is deterministic and auditable. The refusal names the actual defect, so the ledger says why a finding died, not merely that it did.
The trade is deliberate: a comparison answers a much smaller question than a judge does, and it answers it perfectly. Everything the smaller question cannot see is handled by moving work into it — by decoding evidence rather than describing it, so there is something concrete left to compare against.
An accuracy report with no cost column is an advertisement. A tool that refuses everything has perfect precision, so precision on its own is free — the number that has to sit beside it is how much true work was thrown away getting there.
The third line is the one that separates accuracy from luck, and it is the one that found a defect. Fifty-seven claims were being refused correctly and logged for the wrong reason. 1,083 is classes times records, not 1,083 independent experiments — the per-class breakdown in the repo is the honest way to read it.
It is not a bug that got left in. It is the property, doing its job on a case where the property is expensive, and it is the first thing worth putting in front of a reviewer. The same run admits the C2 that is shown by the evidence — an address carried two layers down inside a carved PowerShell command line, reachable only by decoding the stager rather than describing it.
Every identity in the system is derived from content. Records are hash-chained, and the head of the chain is a fingerprint of the case rather than of the moment it was produced — so there is no timestamp, sequence number or random seed anywhere for two runs to disagree about.
Which makes a report checkable by someone who does not trust it. Re-run the case, compare the SHA-256, and any divergence is a real difference in the evidence or the code — never noise. A ledger that shifts between runs cannot be used to contradict anybody, and a forensic artefact that cannot be used to contradict anybody is decoration.
In June I submitted find-evil to the SANS FIND EVIL! hackathon — an IOC-lifecycle layer that catches a forensic agent's hallucinated indicators. It didn't place. Reading the entries that did, the ones I looked at had reached for the same defence I had: put a second language model in front of the first and have it argue. That is the part I now think is wrong, and this is the rebuild from scratch — same problem, an oracle that cannot be talked round.
Roughly 5,000 lines, standard library only, enforced by a test that parses every shipped module and fails on any import that is not local or stdlib. 349 tests, including one that copies the repo to a temp directory and runs the demo with an isolated interpreter — because a clone-and-run claim in a README is worth exactly as much as the test that keeps it true.