We built a proof pack so judges could check our numbers. Auditing it ourselves found a wrong p-value, a fabricated source, and a cherry-picked run.
In the last post the lesson was prove the metric does something before you optimize it. This one is the same discipline pointed somewhere more uncomfortable: at the artifact I built to be honest.
With the hackathon deadline close, the temptation was to bolt on a flashy feature. The measurement from the earlier logs said otherwise. Aftershock was already feature-complete; what it lacked wasn’t capability, it was legibility — a judge couldn’t quickly check that the numbers were real. So the final push wasn’t features. It was proof: a Decision Receipt that chains any contested ruling (kernel decision → the agent’s own stated rationale → cost → recorded outcome), confidence intervals and a sign test on the benchmark, and the capstone — a one-page Evidence Pack whose entire promise is: every number here traces to a file in the repo. Go check.
It’s the judge’s dream fact-check target. So before I shipped it, I did the judge’s job. I ran a
multi-agent adversarial pass that re-derived every figure in the pack from its source — the
results.json, the run manifests, the scenario JSON — and asked one question per number: does this
actually trace?
The honest headline: 42 of 47 figures matched exactly. The mean lives (society 103.2, swarm 75.6), the costs, the lives-per-dollar, the NYC-Ida demand and latency baseline (948s, 16.5% held) — all clean. But the five that didn’t trace are the whole point of this post, because the document that failed them was the one I wrote specifically so it couldn’t.
The artifact that dares you to check it
The Evidence Pack opens with a line I was proud of: “Every number below traces to a file in this
repo — no figure is asserted without a source path.” Each claim sits next to its source: a results
file, a run.json field, a path inside scenario.json. That framing is a commitment. It also means
a single wrong source citation doesn’t just look sloppy — it detonates the whole document, because a
judge who runs jq on the cited field and gets back null will stop trusting every other number on
the page.
So that’s exactly what the audit checked: not “is the value plausible,” but “if a skeptic runs the command next to this number, do they get this number back?”
So I checked it — and three citations lied
| Claim | The pack cited | The source actually says |
|---|---|---|
| swarm vs scripted — significance | p = 0.375 | p = 0.0625 (swarm loses 5/5; 0.375 was the society row, copy-pasted) |
NYC-Ida pack_digest | "in scenario.json" | not a field there — jq .pack_digest → null (it's a sha256 computed at load) |
| the "demand real, outcomes simulated" caveat line | emitted by the loader | a constant in web.py, chosen per-pack at serve time |
| real incident count path | reference.n_incidents | reference.aggregates.n_incidents (value 2,212 correct) |
None of these flattered the project — the wrong p-value actually understated how cleanly the
swarm loses (0.0625 is the strongest verdict five seeds can give; 0.375 was a number that wandered in
from the adjacent row). The pack_digest value was even correct — it just pointed at the wrong
place to find it. But that’s the trap with a proof artifact: honest intent plus a wrong pointer reads
exactly like a lie to someone checking. A judge running jq .pack_digest scenario.json, getting
null, and concluding the whole pack is hand-waved would be right to. Every one of these got a
corrected source — the digest now cites the code that computes it and the run.json field that
stores it; the p-value reads 0.0625.
A small, sharp aside: the AI reviewers I’d used to pressure-test the plan made the same class of error. One confidently recommended I “ship the scenario pipeline” — a thing already shipped weeks earlier (caught by checking the disk). Another quoted a flattering conformance figure of 0.915; the real number is 0.759 — it had been unable to read the gitignored file and filled the gap with something plausible. Models hallucinate sources too. The defense is identical: trace it.
The one that stung: a flagship at 95% conformance that saved 8 of 90 lives
The traceability misses were fixable typos. This one was a judgment failure, and it’s the reason the post exists.
The pack showcased seed91-society — the society running the real NYC Hurricane Ida scenario — as the
flagship demo, headlined by its doctrine conformance of 0.95: “Qwen follows the structured
doctrine 95% of the time over a full scenario.” True number. Real signal. And next to it, the pack
said nothing about how the run actually went.
| Signal | Value |
|---|---|
| doctrine conformance | 0.952 (cited) |
| lives saved / lost | 8 / 82 (8 of ~90 — not cited) |
| missions failed | 3 (not cited) |
Conformance measures whether agents follow their playbook. It says nothing about whether they
win. Under Ida’s real overwhelming demand, this run followed doctrine almost perfectly and still
lost 82 of 90 lives. Both facts are true and they don’t conflict — but putting “95% disciplined” in
the spotlight and leaving “saved 8 of 90” off the page implies success the run never had. A sharp
judge who opened the run.json would find 8/82 and, fairly, distrust the rest.
This is the cherry-pick failure mode, and it’s seductive precisely because every individual number is real. You don’t have to fabricate anything to mislead — you just have to be selective about which true number gets the headline. The fix wasn’t to bury the run; it was to state it whole: a caveat now sits directly beside it — “conformance measures instruction-following, not lives; this run scored 0.95 but saved only 8 of 90 under heavy Ida demand.” That sentence makes the pack stronger, not weaker, because it’s the sentence a skeptic was going to write for me.
It wasn’t only the pack
Once I started auditing, the pattern repeated across every honesty surface I’d built that week:
- The Decision Receipt labeled the commander’s rulings “decided by kernel.” The commander is an LLM arbiter, not the deterministic engine — calling its calls “kernel” decisions is exactly the kind of authority-laundering the receipt exists to prevent.
- It showed a ruling as GRANTED when the auction had accepted the bid but the kernel then rejected the dispatch downstream — implying a resource reached a mission it never reached.
- It printed a town-wide lives-saved delta inside a card scoped to a single ruling, so opening the receipt for a declined call could show it next to “+1 saved.”
Three honesty bugs in the component whose entire job is honesty. All caught the same way: an independent pass that didn’t trust the summary and went to the source. All fixed before any of it reached a judge.
The takeaway
Aftershock’s thesis, from Log 001, has been one sentence: don’t trust the summary — check the receipts. It’s the argument for the whole project: typed decisions over chat transcripts, paired seeds over a single triumphant run, provenance labels over a confident demo.
This week that argument turned around and pointed at me. I built the most honest artifact I could — a pack that invites fact-checking — and it still shipped a wrong p-value, a citation to a field that doesn’t exist, and a flagship that omitted its own outcome. Not from bad faith. From the ordinary gravity that pulls every summary toward looking better than the thing it summarizes.
The lesson is narrow and, I think, the most useful one of the whole build:
Intending to be honest is not the same as being honest. A proof artifact is itself a claim, and claims need an adversary. The Evidence Pack only became trustworthy after something tried to break it — and the thing that broke it wasn’t more good intentions, it was the same boring discipline as the ruler in Log 004: re-derive the number from the source, and believe only what survives.
If you build a system to keep agents honest, point it at yourself last. That’s where it finds the most.
Try it live: https://aftershock.redoubtlabs.dev · Read the code: https://github.com/bluntmachetti/aftershock
(the corrected proof bundle is
docs/EVIDENCE.md;
the receipt is web/src/components/DecisionReceipt.tsx)
Built with Qwen Cloud (qwen3.5-flash / qwen3.5-plus / qwen3-max via DashScope) and Alibaba
Cloud ECS, for the Qwen Cloud Global AI Hackathon.