Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

Three Dead Ends in an AI Pairing Session, and the Replayable Claim Ledger That Ended It

Description and use

During an AI pairing session, engineers fixed a double‑charge bug but the assistant’s explanatory claims lacked evidence. They introduced a claim ledger, recording each claim with a command that verifies it, enabling the senior engineer to accept the patch.

Problem

A webhook handler double‑charged customers due to a retry bug, and a coding assistant’s patch was rejected because its explanatory claims were not verifiable.

Outcome

The claim ledger turned prose assertions into executable tests, enabling the senior engineer to verify all claims and accept the patch.

Workflow

  • 1. Reproduce the double‑charge bug by sending a signed webhook twice.
  • 2. Use the AI assistant to propose a patch and an explanation with four claims.
  • 3. Attempt to review the explanation by reading the transcript, re‑rolling the prompt, and relying on a passing test suite—each approach fails to provide evidence for all claims.
  • 4. Create a JSONL claim ledger, recording each claim with a command that will fail if the claim is false.
  • 5. Run the ledger runner against a pinned revision to replay each claim and record verdicts.

Evidence

The bug was small: a webhook handler that double‑charged on retried deliveries.

Source

The assistant proposed an idempotency key, a lookup before insert, and a small migration. Its explanation contained four assertions:…

Source

The decision that survived: a claim ledger… Before the next patch, the pair wrote every assertion the assistant made into a JSONL file…

Source

  • Record date: 2026-10-08T11:10:51.510Z