Sofia/DB-cleanup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

feat(sources): Wayback pass, automatic trimming, detention rule, ten cases re-sourced

Browse Source 
Second and third rounds on the dossier pipeline, reviewed locally before commit.

Retrieval
- wayback_pass.py: retries every BLOCKED/DEAD/THIN/ERROR linked source through
  the Wayback Machine; blocks carry the snapshot date, archive URL and the live
  verdict. 143 slots retried: 93 fetched, 37 no snapshot, 12 script shells, 1 PDF.
- fetch_sources.py: fetch_wayback() helper.
- add_extra_sources.py: files staged extra_* finds (web search, not on the DB
  record) into the store with a provenance note.
- Cases re-sourced by web search: 248 (Oslo: Document.no, Avisa Oslo, NRK; the
  linked Le Parisien piece is case 242 and is marked OFF-CASE), 258 Kharkiv,
  260 Singapore, 358 Bangkok, 365 Las Vegas, 388 Verneuil-sur-Seine, 390 Zoersel,
  430 Homestead. 341 of 364 cases now hold a fetched article; 11, 66 and 399
  have no public text source (podcast, police video, direct victim report).

Trimming
- auto_trim.py: case-anchored furniture cut for UNTRIMMED blocks. Finds the body
  run that mentions the case, merges across subheadings and short furniture
  gaps, drops teasers, share bars, date/URL/caption lines and subscription
  pitches; refuses pages with no record term or almost no body. 302 blocks
  trimmed; 16 left on auto-trim-review.md (7 OFF-CASE suspects). Every cut is
  labelled AUTO-TRIMMED, UNREVIEWED in the dossier; the full extract stays in
  staging/. Decisions with anchors in trim-decisions-auto.json.
- build_cases.py / verify_all.py: banners for auto-trimmed and archived blocks,
  staged-file check extended to Wayback blocks, flag legend under the record.

Detention rule
- README "Definitions": the DB field `kidnappings` is the DETENTION violence
  type (victim, guard, staff or relative held to force submission or execute
  the theft), distinct from the Kidnapping scenario (taken away and held).
- detention-flag-review.md / detention-flag-corrections.json: 40 records
  reviewed with evidence; 30 set-to-1 proposals accepted by the owner on
  2026-09-06 (listed in corrections-approved.md), 10 still open.
- apply_detention_wording.py: "Violence Used" in the 70 reviewed summaries now
  names detention explicitly (62 of 70 labelled), supported by the summary's
  own text; idempotent; supersedes the batch scripts' wording.

Worklist and docs
- sofia-worklist.md: 248 decisions, detention rule item replacing the old
  "no abduction" item, fresh-search section for the textless cases.
- README-START-HERE.md: progress notes, run commands, next steps.
- Bug fixed in passing: Wayback blocks stored in-memory text with carriage
  returns; now stored as read back from disk.

verify_all.py PASSES (2321 checks).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
This commit is contained in:
StellarCrow
2026-09-06 19:36:45 +02:00
co-authored by Claude Fable 5.1
parent 499fd61f79
commit 439e83a4cb
588 changed files with 24800 additions and 13190 deletions
Download Patch File Download Diff File Expand all files Collapse all files
merge-output/sources/203_Influencer-s-father.txt
+19 -97
View File
@@ -27,6 +27,9 @@ DATABASE RECORD
has_processed_date 1
created_at 2025-11-13 05:44:45
(flag legend: 'kidnappings' is the DETENTION flag — 1 when the victim was held against
their will at any point, tied, locked in, held at gunpoint or taken away; it is NOT the
same as the 'Kidnapping' scenario, which means taken away and held. See README, Definitions.)
AI SUMMARY [generated at bulk import, NOT verified against sources]
***Victims:** Father (56 years old) and mother of Dubai-based cryptocurrency influencer/entrepreneur
@@ -86,19 +89,19 @@ Crypto ransom demands, a rising phenomenon
Those working in this sector are even more targeted for kidnappings with ransom demands as cryptocurrencies are the subject of all fantasies. Moreover, criminals know that crypto can be transferred from one end of the world to the other: if they can't hijack luxury cars in Dubai, they can receive crypto on a wallet from France.
------------------------------------------------------------------------------------------------
FETCHED ARTICLE 1 [RAW EXTRACT — NOT YET TRIMMED]
!! This is the whole page as extracted. It still contains site furniture,
!! and may contain teaser headlines for UNRELATED cases. Trim on read-through.
!! Do not quote from it without checking the passage belongs to this case.
FETCHED ARTICLE 1 [AUTO-TRIMMED — UNREVIEWED]
!! Furniture was cut by a script keyed on this case's record, not by a person.
!! It may still carry passages about OTHER cases from a round-up article.
!! The full page extract is kept in staging/ if a cut needs revisiting.
Source cryptoast.fr
URL https://cryptoast.fr/pere-influenceur-crypto-sequestre-soir-nouvel-an/
Field url
Retrieved 2026-09-05T12:49:20+00:00 via raw HTTP retrieval, deterministic extraction (HTTP 200)
Language unknown
Kept paragraphs ALL 60 paragraphs, untrimmed
Length 6,704 chars
Integrity sha256 352acd712029172a9f2bf684eb48a8c07ae55baed71fb224cd27039533e3ebaa
Trimmed NOT TRIMMED. Full page extract; trim on read-through.
Kept paragraphs headline [0] + 16-55 excl [17, 21] of 60
Length 6,048 chars
Integrity sha256 8abab232344a4f9be68d7d987ba9d639ca7f249374866baff43bd4a5fc47d87e
Trimmed AUTO-TRIMMED, unreviewed: kept paragraphs 16-55 plus headline [0]; dropped 15 leading and 4 trailing paragraphs and 2 interior furniture line(s). Case-term hits on page: 22.
Caveat retrieved from a live page on the date above. Unlike text copied
from the spreadsheet, there is no second copy to hash it against.
------------------------------------------------------------------------------------------------
@@ -107,69 +110,14 @@ ORIGINAL (unknown, verbatim as retrieved)
Le père d'un influenceur crypto a été séquestré le soir du nouvel an
Le père d'un influenceur crypto a été séquestré le soir du nouvel an
Aucun résultat trouvé.
Connectez-vous pour gagner des TOASTS, accomplir des quêtes et débloquer des badges.
Accueil » Actu de la crypto » Actualité du Bitcoin (BTC) » Le père d'un influenceur crypto a été séquestré le soir du nouvel an
Le père d'un influenceur crypto a été séquestré le soir du nouvel an
Dans la nuit du nouvel an, une nouvelle histoire d'enlèvement avec demande de rançon en crypto a eu lieu. Pour viser un influenceur vivant à Dubaï, les criminels n'ont pas hésité à séquestrer son père, à le rouer de coup et à l'asperger d'essence. Les crimes visant les détenteurs de cryptomonnaies sont-ils en train d'exploser ?
le 3 janvier 2025 à 16:30.
https://cryptoast.fr/pere-influenceur-crypto-sequestre-soir-nouvel-an/
3 commentairesRéagir3Ajoutez-nous à vos favorisAjoutez-nous
Un homme de 50 ans retrouvé dans le coffre d'une voiture
@@ -239,28 +187,20 @@ Bitcoin dévisse de 1 600 $ en trois minutes à cause des chiffres US
Prix du Bitcoin en septembre : le spectre de « Rektember »
Bitcoin : le hashrate traverse son premier « bear market » face l’IA, selon Twenty One Capital
A la suite de votre inscription et selon le montant du dépôt
Pour un dépôt de 100 € à la suite de votre inscription
S'inscrire avec le code CRYPTOAST et trader au moins 1 000 € avant le 30/09
------------------------------------------------------------------------------------------------
FETCHED ARTICLE 2 [RAW EXTRACT — NOT YET TRIMMED]
!! This is the whole page as extracted. It still contains site furniture,
!! and may contain teaser headlines for UNRELATED cases. Trim on read-through.
!! Do not quote from it without checking the passage belongs to this case.
FETCHED ARTICLE 2 [AUTO-TRIMMED — UNREVIEWED]
!! Furniture was cut by a script keyed on this case's record, not by a person.
!! It may still carry passages about OTHER cases from a round-up article.
!! The full page extract is kept in staging/ if a cut needs revisiting.
Source dlnews.com
URL https://www.dlnews.com/articles/people-culture/french-teen-helped-kidnap-crypto-influencers-father-police/
Field url_2
Retrieved 2026-09-05T12:49:22+00:00 via raw HTTP retrieval, deterministic extraction (HTTP 200)
Language unknown
Kept paragraphs ALL 39 paragraphs, untrimmed
Length 4,502 chars
Integrity sha256 4784539dcf92552132f4509bc54a0b4cc10cfa1b839666abe56892c737db16cf
Trimmed NOT TRIMMED. Full page extract; trim on read-through.
Kept paragraphs headline [0] + 3-31 of 39
Length 3,881 chars
Integrity sha256 e88509b692c5f60393e8b5cf8ffc1280528553c0e1c6e1f589ed246cd70df5b1
Trimmed AUTO-TRIMMED, unreviewed: kept paragraphs 3-31 plus headline [0]; dropped 2 leading and 7 trailing paragraphs. Case-term hits on page: 35.
Caveat retrieved from a live page on the date above. Unlike text copied
from the spreadsheet, there is no second copy to hash it against.
------------------------------------------------------------------------------------------------
@@ -269,10 +209,6 @@ ORIGINAL (unknown, verbatim as retrieved)
Teen kidnapped crypto influencer’s dad and doused him in petrol, police say - DL News
This article is more than three months old
Teen kidnapped crypto influencer’s dad and doused him in petrol, police say
French teen helped kidnap crypto influencer’s father - police. Illustration: Andrés Tapia; Source: Shutterstock.
HomePeople & CultureTeen kidnapped crypto influencer’s dad and doused him in petrol, police say
@@ -331,20 +267,6 @@ After they triggered an alarm, the group allegedly fled the scene. Police say th
Tim Alper is a News Correspondent at DL News. Got a tip? Email him at tdalper@dlnews.com.
How cowboy Bitcoin miners got caught stealing energy from Russian village
They said they were police, but pulled a knife and stole $1m in Bitcoin
Scammers prey on pro-Iran crypto donors, warn politicians
Crypto crime on the rise in Spain’s Basque Country, say police
Crypto thief teens trigger helicopter manhunt after bomb threat
Crypto CEO shows ‘insufficient remorse’ after throwing girlfriend out 19th floor
French police arrest 12 as probe into spate of violent crypto kidnappings widens
================================================================================================
generated from attacks-export-Gart-website.json + reported_K&R | case 203
================================================================================================