Archived
docs(verify): consolidated verification report and README update
make_verification_report.py builds batches/VERIFICATION-REPORT.md (coverage, contradictions by field, flag proposals, label frequency, cases no label fits, label gaps by theme, scope/merge/split decisions) and batches/contradictions-ALL.md from the applied batch files. README progress and quick-start rewritten for the finished verification; the old enrichment method is kept for reference. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
9470e6bb0c
commit
c9d6a0f2fd
+31
-28
@@ -109,7 +109,7 @@ Two different kinds of change came out of this work. They are NOT the same:
|
||||
- verify_all.py PASSES (896 checks). Confirmed from this bundle's new location.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
## The task in progress: summary enrichment
|
||||
## The summary enrichment method (finished 2026-09-30; kept for reference)
|
||||
|
||||
Goal: bring the short/weak bulk-import summaries up to the standard of the newest cases
|
||||
(447-462), which have six labelled categories and run roughly 970-2100 characters:
|
||||
@@ -170,21 +170,24 @@ foiled plots, and any known wrong-source case so it does not over-flag."
|
||||
--------------------------------------------------------------------------------
|
||||
## Progress
|
||||
|
||||
Done and reviewer-verified: batches 1-7, 70 of 256 cases. IDs completed:
|
||||
B1: 372,371,370,369,368,367,365,364,362,353
|
||||
B2: 350,349,340,334,323,322,321,317,315,312
|
||||
B3: 309,308,307,304,303,301,299,298,297,296
|
||||
B4: 293,292,290,288,285,279,273,272,271,270
|
||||
B5: 266,263,256,253,252,249,248,247,246,245
|
||||
B6: 244,243,242,240,238,237,236,235,234,233
|
||||
B7: 232,231,230,229,228,224,222,221,219,218
|
||||
2026-09-30: every case in the export has a verified summary and a scenario sequence, except
|
||||
where noted in merge-output/batches/VERIFICATION-REPORT.md. The 70 cases done by hand
|
||||
(IDs 218-372, batches B1-B7 of the old method) were kept; the remaining 319 were verified in
|
||||
32 batches of ten (merge-output/batches/batch-NN.json) per VERIFICATION-PROCEDURE.md: an
|
||||
opus writer per batch, a read-only sonnet checker per batch, fixes applied, then
|
||||
apply_batch.py. summary-overrides.json and scenario-sequences.json hold one entry per case.
|
||||
verify_all.py PASSES.
|
||||
|
||||
summary-overrides.json holds 70 entries. verify_all.py PASSES.
|
||||
|
||||
Remaining: continue descending from ID 217 (217,216,215,214,213,212,211,210,208,207 ...) down to
|
||||
case 1 - about 186 short cases, roughly 19 more batches. Get the exact remaining list with:
|
||||
python ".../merge-output/survey_summaries.py"
|
||||
and take the ids flagged SHORT / unstructured, descending.
|
||||
Deliverables for Sofia (all proposals; the DB export is untouched):
|
||||
- merge-output/batches/VERIFICATION-REPORT.md: coverage, contradiction counts by field,
|
||||
flag proposals, label frequency, the cases no label fits, label gaps by theme, and the
|
||||
scope/merge/split decisions. Regenerate with make_verification_report.py after any apply.
|
||||
- merge-output/batches/contradictions-ALL.md: every record-vs-article contradiction with the
|
||||
quoted article line, merged from contradictions-01..32.md.
|
||||
- merge-output/batches/flag-proposals.json: every flag the file does not support, with the
|
||||
proposed value and evidence line.
|
||||
- merge-output/exports/gart-kr-database-<date>.xlsx: the review sheet (export_xlsx.py).
|
||||
- merge-output/sofia-worklist.md: manual fetch list, case notes, batch pointers.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
## 2026-09-05 update: linked URLs fetched for the 252 spreadsheet-only cases
|
||||
@@ -253,16 +256,16 @@ file; remove the stale one or verify_all.py reports one file too many.
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
## Quick-start checklist for the next AI
|
||||
1. Read this file and merge-output/sofia-worklist.md.
|
||||
2. Confirm the bundle is healthy:
|
||||
python ".../merge-output/verify_all.py" (should say PASS)
|
||||
summary-overrides.json should have 70 entries.
|
||||
3. Pick the next ~10 short cases descending from ID 217 (use survey_summaries.py). Read each
|
||||
dossier in full. Write build_overrides_batch8.py (copy batch 7's structure). Run it, then
|
||||
build_cases.py --all, then verify_all.py.
|
||||
4. Run the source-checker agent on the batch (read-only, no fetch). Apply fair flags; rebuild;
|
||||
verify. Then the batch is locked.
|
||||
5. Repeat down to case 1.
|
||||
6. Keep adding DB field/source problems to sofia-worklist.md. Never edit the DB.
|
||||
7. When all summaries are done, the deliverables are: the sources/*.txt (already carry the
|
||||
improved summaries) and sofia-worklist.md (Sofia's manual DB fixes, done last on her side).
|
||||
1. Read this file, merge-output/DEFINITIONS.md and merge-output/VERIFICATION-PROCEDURE.md.
|
||||
2. Confirm the bundle is healthy: python merge-output/verify_all.py (should say PASS);
|
||||
summary-overrides.json and scenario-sequences.json should have one entry per DB case.
|
||||
3. New cases: python merge-output/pull_live_cases.py, then fetch (stage_all.py
|
||||
--include-sheet-cases, wayback_pass.py, auto_trim.py), build_cases.py --all, verify_all.py.
|
||||
Then verify them as a new batch: add the ids to batches/batch-lists.json, run a writer
|
||||
agent with batches/WRITER-PROMPT.md, a read-only checker, batches/validate.py,
|
||||
apply_batch.py, make_verification_report.py, export_xlsx.py.
|
||||
4. Never edit the DB export or the xlsx sheet by hand. Every correction is a proposal in
|
||||
batches/ or sofia-worklist.md until Sofia applies it.
|
||||
5. Commit on branch fetch-sheet-only-sources with the StellarCrow identity; main holds only
|
||||
the handoff bundle and DEFINITIONS.md. Gitea pushes may print "Terminated" but land:
|
||||
confirm with git ls-remote --heads origin fetch-sheet-only-sources.
|
||||
|
||||
Reference in New Issue
Block a user