docs(verify): consolidated verification report and README update

make_verification_report.py builds batches/VERIFICATION-REPORT.md (coverage,
contradictions by field, flag proposals, label frequency, cases no label fits,
label gaps by theme, scope/merge/split decisions) and batches/contradictions-ALL.md
from the applied batch files. README progress and quick-start rewritten for the
finished verification; the old enrichment method is kept for reference.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
This commit is contained in:
StellarCrow
2026-09-30 12:10:51 +02:00
co-authored by Claude Fable 5.1
parent 9470e6bb0c
commit c9d6a0f2fd
4 changed files with 3954 additions and 28 deletions
+31 -28
View File
@@ -109,7 +109,7 @@ Two different kinds of change came out of this work. They are NOT the same:
- verify_all.py PASSES (896 checks). Confirmed from this bundle's new location.
--------------------------------------------------------------------------------
## The task in progress: summary enrichment
## The summary enrichment method (finished 2026-09-30; kept for reference)
Goal: bring the short/weak bulk-import summaries up to the standard of the newest cases
(447-462), which have six labelled categories and run roughly 970-2100 characters:
@@ -170,21 +170,24 @@ foiled plots, and any known wrong-source case so it does not over-flag."
--------------------------------------------------------------------------------
## Progress
Done and reviewer-verified: batches 1-7, 70 of 256 cases. IDs completed:
B1: 372,371,370,369,368,367,365,364,362,353
B2: 350,349,340,334,323,322,321,317,315,312
B3: 309,308,307,304,303,301,299,298,297,296
B4: 293,292,290,288,285,279,273,272,271,270
B5: 266,263,256,253,252,249,248,247,246,245
B6: 244,243,242,240,238,237,236,235,234,233
B7: 232,231,230,229,228,224,222,221,219,218
2026-09-30: every case in the export has a verified summary and a scenario sequence, except
where noted in merge-output/batches/VERIFICATION-REPORT.md. The 70 cases done by hand
(IDs 218-372, batches B1-B7 of the old method) were kept; the remaining 319 were verified in
32 batches of ten (merge-output/batches/batch-NN.json) per VERIFICATION-PROCEDURE.md: an
opus writer per batch, a read-only sonnet checker per batch, fixes applied, then
apply_batch.py. summary-overrides.json and scenario-sequences.json hold one entry per case.
verify_all.py PASSES.
summary-overrides.json holds 70 entries. verify_all.py PASSES.
Remaining: continue descending from ID 217 (217,216,215,214,213,212,211,210,208,207 ...) down to
case 1 - about 186 short cases, roughly 19 more batches. Get the exact remaining list with:
python ".../merge-output/survey_summaries.py"
and take the ids flagged SHORT / unstructured, descending.
Deliverables for Sofia (all proposals; the DB export is untouched):
- merge-output/batches/VERIFICATION-REPORT.md: coverage, contradiction counts by field,
flag proposals, label frequency, the cases no label fits, label gaps by theme, and the
scope/merge/split decisions. Regenerate with make_verification_report.py after any apply.
- merge-output/batches/contradictions-ALL.md: every record-vs-article contradiction with the
quoted article line, merged from contradictions-01..32.md.
- merge-output/batches/flag-proposals.json: every flag the file does not support, with the
proposed value and evidence line.
- merge-output/exports/gart-kr-database-<date>.xlsx: the review sheet (export_xlsx.py).
- merge-output/sofia-worklist.md: manual fetch list, case notes, batch pointers.
--------------------------------------------------------------------------------
## 2026-09-05 update: linked URLs fetched for the 252 spreadsheet-only cases
@@ -253,16 +256,16 @@ file; remove the stale one or verify_all.py reports one file too many.
--------------------------------------------------------------------------------
## Quick-start checklist for the next AI
1. Read this file and merge-output/sofia-worklist.md.
2. Confirm the bundle is healthy:
python ".../merge-output/verify_all.py" (should say PASS)
summary-overrides.json should have 70 entries.
3. Pick the next ~10 short cases descending from ID 217 (use survey_summaries.py). Read each
dossier in full. Write build_overrides_batch8.py (copy batch 7's structure). Run it, then
build_cases.py --all, then verify_all.py.
4. Run the source-checker agent on the batch (read-only, no fetch). Apply fair flags; rebuild;
verify. Then the batch is locked.
5. Repeat down to case 1.
6. Keep adding DB field/source problems to sofia-worklist.md. Never edit the DB.
7. When all summaries are done, the deliverables are: the sources/*.txt (already carry the
improved summaries) and sofia-worklist.md (Sofia's manual DB fixes, done last on her side).
1. Read this file, merge-output/DEFINITIONS.md and merge-output/VERIFICATION-PROCEDURE.md.
2. Confirm the bundle is healthy: python merge-output/verify_all.py (should say PASS);
summary-overrides.json and scenario-sequences.json should have one entry per DB case.
3. New cases: python merge-output/pull_live_cases.py, then fetch (stage_all.py
--include-sheet-cases, wayback_pass.py, auto_trim.py), build_cases.py --all, verify_all.py.
Then verify them as a new batch: add the ids to batches/batch-lists.json, run a writer
agent with batches/WRITER-PROMPT.md, a read-only checker, batches/validate.py,
apply_batch.py, make_verification_report.py, export_xlsx.py.
4. Never edit the DB export or the xlsx sheet by hand. Every correction is a proposal in
batches/ or sofia-worklist.md until Sofia applies it.
5. Commit on branch fetch-sheet-only-sources with the StellarCrow identity; main holds only
the handoff bundle and DEFINITIONS.md. Gitea pushes may print "Terminated" but land:
confirm with git ls-remote --heads origin fetch-sheet-only-sources.