- auto_trim.py: trim-decisions-auto.json and auto-trim-review.md are now derived
from the whole store on every run (paragraphs_kept, cut_note, staged files),
so a run limited to a few cases no longer overwrites the decisions for the
rest. Restores the full trail: 378 decisions across 287 cases, 36 review
entries (7 off-case suspects, 12 too small, 20 weak anchors).
- make_fetch_list.py: regenerates the manual fetch list on sofia-worklist.md
from the store between markers; replaces the hand-written July list. 65
sources remain unreadable by script, none on a case without other text.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
- pull_live_cases.py: merges https://stats.gart.io/api/attacks into the export.
The feed lacks notes/reports/original_date/has_processed_date, so existing
records keep their fields; changed values are taken from live (record 188:
victim, violence_torture, weapons, summary) and new records are appended
with those four fields null. Previous export kept as
attacks-export-Gart-website-2026-09-09-before.json; raw feed saved as
attacks-live-api-2026-09-09.json. Export now has 389 records.
- 25 new dossiers built; 93 linked URL slots fetched (77 fetched, 8 blocked,
4 thin, 4 video/social); 76 blocks auto-trimmed. Wayback retry for the 12
failed slots hit archive.org rate limits (429) and is left for a later run.
- Sheet row 252 (the Phuket robbery listed in cases-missing-from-database.json)
approved as the source for new case 476, which is that event.
- Stale dossier for renamed case 188 removed.
- Detention flag check on the new cases: 479 proposed 0->1 (forced entry plus
assault), 468 null->0 proposed, 483 and 486 confirmed 0; review file and
corrections JSON updated. Worklist and README carry the procedure.
verify_all.py PASSES (2776 checks, 389 case files).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
Owner's decisions of 2026-09-07 applied:
- Rule sharpened in README "Definitions": forced entry into the victim's
space (home, office, hotel room, car) followed by beating or holding them
there to take what is wanted is detention, however short.
- 354, 355 (null) and 124 set to 0. 242, 273, 413, 422, 423, 215, 298 stay 1;
the set-to-0 proposals for them are withdrawn under the rule.
- corrections-approved.md, detention-flag-review.md and the corrections JSON
carry the decisions; the worklist item says no open items remain.
- Summaries 242 and 273 relabelled "Detention" with the forced-entry wording.
verify_all.py PASSES.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
Second and third rounds on the dossier pipeline, reviewed locally before commit.
Retrieval
- wayback_pass.py: retries every BLOCKED/DEAD/THIN/ERROR linked source through
the Wayback Machine; blocks carry the snapshot date, archive URL and the live
verdict. 143 slots retried: 93 fetched, 37 no snapshot, 12 script shells, 1 PDF.
- fetch_sources.py: fetch_wayback() helper.
- add_extra_sources.py: files staged extra_* finds (web search, not on the DB
record) into the store with a provenance note.
- Cases re-sourced by web search: 248 (Oslo: Document.no, Avisa Oslo, NRK; the
linked Le Parisien piece is case 242 and is marked OFF-CASE), 258 Kharkiv,
260 Singapore, 358 Bangkok, 365 Las Vegas, 388 Verneuil-sur-Seine, 390 Zoersel,
430 Homestead. 341 of 364 cases now hold a fetched article; 11, 66 and 399
have no public text source (podcast, police video, direct victim report).
Trimming
- auto_trim.py: case-anchored furniture cut for UNTRIMMED blocks. Finds the body
run that mentions the case, merges across subheadings and short furniture
gaps, drops teasers, share bars, date/URL/caption lines and subscription
pitches; refuses pages with no record term or almost no body. 302 blocks
trimmed; 16 left on auto-trim-review.md (7 OFF-CASE suspects). Every cut is
labelled AUTO-TRIMMED, UNREVIEWED in the dossier; the full extract stays in
staging/. Decisions with anchors in trim-decisions-auto.json.
- build_cases.py / verify_all.py: banners for auto-trimmed and archived blocks,
staged-file check extended to Wayback blocks, flag legend under the record.
Detention rule
- README "Definitions": the DB field `kidnappings` is the DETENTION violence
type (victim, guard, staff or relative held to force submission or execute
the theft), distinct from the Kidnapping scenario (taken away and held).
- detention-flag-review.md / detention-flag-corrections.json: 40 records
reviewed with evidence; 30 set-to-1 proposals accepted by the owner on
2026-09-06 (listed in corrections-approved.md), 10 still open.
- apply_detention_wording.py: "Violence Used" in the 70 reviewed summaries now
names detention explicitly (62 of 70 labelled), supported by the summary's
own text; idempotent; supersedes the batch scripts' wording.
Worklist and docs
- sofia-worklist.md: 248 decisions, detention rule item replacing the old
"no abduction" item, fresh-search section for the textless cases.
- README-START-HERE.md: progress notes, run commands, next steps.
- Bug fixed in passing: Wayback blocks stored in-memory text with carriage
returns; now stored as read back from disk.
verify_all.py PASSES (2321 checks).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
StellarCrowandClaude Fable 5.1
SofiaandClaude Opus 4.8