Sofia/DB-cleanup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

feat(sources): Wayback pass, automatic trimming, detention rule, ten cases re-sourced

Browse Source 
Second and third rounds on the dossier pipeline, reviewed locally before commit.

Retrieval
- wayback_pass.py: retries every BLOCKED/DEAD/THIN/ERROR linked source through
  the Wayback Machine; blocks carry the snapshot date, archive URL and the live
  verdict. 143 slots retried: 93 fetched, 37 no snapshot, 12 script shells, 1 PDF.
- fetch_sources.py: fetch_wayback() helper.
- add_extra_sources.py: files staged extra_* finds (web search, not on the DB
  record) into the store with a provenance note.
- Cases re-sourced by web search: 248 (Oslo: Document.no, Avisa Oslo, NRK; the
  linked Le Parisien piece is case 242 and is marked OFF-CASE), 258 Kharkiv,
  260 Singapore, 358 Bangkok, 365 Las Vegas, 388 Verneuil-sur-Seine, 390 Zoersel,
  430 Homestead. 341 of 364 cases now hold a fetched article; 11, 66 and 399
  have no public text source (podcast, police video, direct victim report).

Trimming
- auto_trim.py: case-anchored furniture cut for UNTRIMMED blocks. Finds the body
  run that mentions the case, merges across subheadings and short furniture
  gaps, drops teasers, share bars, date/URL/caption lines and subscription
  pitches; refuses pages with no record term or almost no body. 302 blocks
  trimmed; 16 left on auto-trim-review.md (7 OFF-CASE suspects). Every cut is
  labelled AUTO-TRIMMED, UNREVIEWED in the dossier; the full extract stays in
  staging/. Decisions with anchors in trim-decisions-auto.json.
- build_cases.py / verify_all.py: banners for auto-trimmed and archived blocks,
  staged-file check extended to Wayback blocks, flag legend under the record.

Detention rule
- README "Definitions": the DB field `kidnappings` is the DETENTION violence
  type (victim, guard, staff or relative held to force submission or execute
  the theft), distinct from the Kidnapping scenario (taken away and held).
- detention-flag-review.md / detention-flag-corrections.json: 40 records
  reviewed with evidence; 30 set-to-1 proposals accepted by the owner on
  2026-09-06 (listed in corrections-approved.md), 10 still open.
- apply_detention_wording.py: "Violence Used" in the 70 reviewed summaries now
  names detention explicitly (62 of 70 labelled), supported by the summary's
  own text; idempotent; supersedes the batch scripts' wording.

Worklist and docs
- sofia-worklist.md: 248 decisions, detention rule item replacing the old
  "no abduction" item, fresh-search section for the textless cases.
- README-START-HERE.md: progress notes, run commands, next steps.
- Bug fixed in passing: Wayback blocks stored in-memory text with carriage
  returns; now stored as read back from disk.

verify_all.py PASSES (2321 checks).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
This commit is contained in:
StellarCrow
2026-09-06 19:36:45 +02:00
co-authored by Claude Fable 5.1
parent 499fd61f79
commit 439e83a4cb
588 changed files with 24800 additions and 13190 deletions
Download Patch File Download Diff File Expand all files Collapse all files
merge-output/sources/123_Pierrick-Jamaux.txt
+11 -88
View File
@@ -26,6 +26,9 @@ DATABASE RECORD
notes Attempt to steal the watch failed, but victim was shot 5 times in the legs. One of the shots hit his femoral artery; almost killed him
has_processed_date 1
created_at 2025-11-13 05:44:45
(flag legend: 'kidnappings' is the DETENTION flag — 1 when the victim was held against
their will at any point, tied, locked in, held at gunpoint or taken away; it is NOT the
same as the 'Kidnapping' scenario, which means taken away and held. See README, Definitions.)
AI SUMMARY [generated at bulk import, NOT verified against sources]
**Victim:** Pierrick Jamaux
@@ -93,19 +96,19 @@ He blamed petty crime and homelessness as contributing factors to the widespread
'Anything goes in the City of New York,' Adams said. 'The most important city on the globe has become the laughingstock of the globe. And the dysfunctionality of our city has cascaded throughout the entire country.'
------------------------------------------------------------------------------------------------
FETCHED ARTICLE 1 [RAW EXTRACT — NOT YET TRIMMED]
!! This is the whole page as extracted. It still contains site furniture,
!! and may contain teaser headlines for UNRELATED cases. Trim on read-through.
!! Do not quote from it without checking the passage belongs to this case.
FETCHED ARTICLE 1 [AUTO-TRIMMED — UNREVIEWED]
!! Furniture was cut by a script keyed on this case's record, not by a person.
!! It may still carry passages about OTHER cases from a round-up article.
!! The full page extract is kept in staging/ if a cut needs revisiting.
Source dailymail.co.uk
URL https://www.dailymail.co.uk/news/article-10663677/Crypto-expert-33-shot-robber-targeting-450k-watch-speaks-out.html
Field url
Retrieved 2026-09-05T12:52:11+00:00 via raw HTTP retrieval, deterministic extraction (HTTP 200)
Language unknown
Kept paragraphs ALL 100 paragraphs, untrimmed
Length 13,486 chars
Integrity sha256 7be38bbf8646f2d7f76c902cc7729684a754164de69fbd9ebec8e1e71cbd59ac
Trimmed NOT TRIMMED. Full page extract; trim on read-through.
Kept paragraphs headline [0] + 32-90 of 100
Length 8,439 chars
Integrity sha256 f1b613b2a6f5e6235cd877dbfe94d70291c8b841d049f643d2193e7db2868816
Trimmed AUTO-TRIMMED, unreviewed: kept paragraphs 32-90 plus headline [0]; dropped 31 leading and 9 trailing paragraphs. Case-term hits on page: 69.
Caveat retrieved from a live page on the date above. Unlike text copied
from the spreadsheet, there is no second copy to hash it against.
------------------------------------------------------------------------------------------------
@@ -114,68 +117,6 @@ ORIGINAL (unknown, verbatim as retrieved)
Crypto expert, 33, shot by robber targeting his $450k watch speaks out | Daily Mail Online
Three hours in Southampton that exposed 'two-tier' Britain: What really happened the night Henry Nowak was murdered in unrivalled DEEP DIVE detail
Police chief apologises to Henry Nowak's family for officers handcuffing and arresting 18-year-old while he bled to death
'This is nothing to do with the faith': Moment Sikh taxi driver shakes hands with Henry Nowak protesters in Southampton
Female ex-police officer is forced to move to a safe house after being wrongly accused of arresting Henry Nowak
DAN HODGES: When Britain needed a leader to articulate our pain, disgust and despair, Keir Starmer was missing in action. Kemi stepped up - and spoke like a real Prime Minister
BBC forced into grovelling apology to Nigel Farage after Newsnight row in which Reform chief accused corporation of 'defamation' over Henry Nowak murder response
Ralf Schumacher's ex-wife aims thinly veiled dig at him over F1's first gay wedding, as new photos emerge of Michael's brother, 50, marrying 36-year-old lover - and lavish after-party
Katie Price's prison showdown with husband revealed: Truth about star's dash to Dubai as insiders tell ALEX DOYLE about her new bust-up with sister Sophie... and how it's affected their mother's health
I've discovered one of the single biggest triggers for breast cancer. It raises your risk of the disease by 60 per cent... but no one is talking about it. Here's everything you need to know, revealed by a leading oncologist
Katie Price looks in despair as she squeezes in a spot of sunbathing in her tiny bikini before embarking on mission to free her husband Lee Andrews from prison
Brad Pitt is stunned by speed of Isle of Man TT as he arrives to start filming at the world's most dangerous race
I know why Henry Nowak was treated as a criminal as he lay dying, while the man who stabbed him was coddled. I've been saying the same thing for 20 years - this is what we must do: PETER HITCHENS
Are YOU rich? Our calculator reveals how your salary stacks up against the UK - and your town and job
Teenage Norwegian hitman flew to Britain to carry out assassination for Iran-linked Swedish gang, court hears
Kate Ferdinand's stepson Tate opens up about their relationship and says he 'never thought' he'd love her children
I was so overweight, I didn't realise I was five months pregnant. Now, I'm down to a size 8 without using Mounjaro. This is the simple plan I followed to lose the weight... and how I've kept it off: BROOKE MILLS
Drinks on you, Ange? Rayner enjoys £61,500 payday as she receives advance on her 'unvarnished and upfront' memoir
Disturbing reason so many are risking their health by injecting the new, unlicensed Godzilla of fat jabs: Black market 'Reta' is everywhere... this is how it supercharges your weight loss - and its brutal side-effects
Three Royal Navy crew members die as Merlin helicopter crashes into Devon field between two military bases, First Sea Lord confirms
Princess Kate pays tribute to three Royal Navy crew members who died in Merlin helicopter crash on training exercise - as investigators continue to probe cause of tragedy
Pakistanis who gang-raped French tourist in front of her three children after her car ran out of fuel will be executed, court rules
Killjoy councils clamp down on mothers selling brownies in their driveways - with 'cake shed' owners forced to pay £1,000 to operate
More than thirty Chinese made Jaecoo SUVs worth nearly £1million go up in flames at Southampton docks
How Super El Niño will wreak havoc around the globe: Step-by-step graphic reveals how the unusual climate event will bring extreme heat 'nearly EVERYWHERE'
EXCLUSIVE: 'New York isn't safe for anyone': Crypto expert, 33, reveals he was shot FIVE times by robber who tried to steal his $450,000 Richard Mille watch after Manhattan hotel ambush
Pierrick Jamaux, 33, was getting out of an Uber in Midtown when he was shot five times on March 18
He told DailyMail.com he believes he was targeted and followed for hours or days before being ambushed
Jamaux said the bullets broke his femur and severed his femoral artery, almost killing him
At the time of the robbery he was wearing a Richard Mille watch, which is worth an eye-watering $450,000
The suspect was unable to wrangle the pricey watch from Jamaux's wrist because its anti-theft mechanism
Jamaux has undergone six surgeries so far but remains in hospital in New York as he recovers from his injuries
Attack comes amid a terrifying surge in crime which the mayor says has made the city a 'laughingstock'
See more Daily Mail on Google - save us as a Preferred Source
@@ -294,24 +235,6 @@ Woman chases off car thief while wearing nothing but a bath towel
Balaclava-clad protesters blockade Dover 'in anti-migrant demo'
Moment truck smashed to pieces by speeding passenger train
Craig Melvin calls out Today co-host live for oversleeping
Judge in the Lindsay Clancy murder trial announces a mistrial
Shock and controversy as judge declares Lindsay Clancy mistrial
Moment brave Montana woman protects dogs from moose in yard
Trump predicts what's next in the Lindsay Clancy murder trial
The comments below have been moderated in advance.
The views expressed in the contents above are those of our users and do not necessarily reflect the views of MailOnline.
We are no longer accepting comments on this article.
================================================================================================
generated from attacks-export-Gart-website.json + reported_K&R | case 123
================================================================================================