feat(sources): fetch linked URLs for the 252 spreadsheet-only cases

Until now only cases 357+ (plus a few older ones) had their linked URLs
fetched; the other 252 dossiers held just the text pasted into the
spreadsheet, often only the lede. This pass fetches every linked URL for
those cases so the full text is on file before link rot takes it.

- stage_all.py: new --include-sheet-cases flag that lifts the skip on
  cases holding spreadsheet articles. Resumable as before.
- staging/: raw extracts and index.json for the 252 cases.
- sourced-articles.json, case-files-manifest.json: 188 new UNTRIMMED
  blocks. 179 cases gained at least one fetched article; 73 got none
  (49 BLOCKED, 18 THIN, 9 DEAD, 4 video/social, 3 ERROR).
- sources/*.txt: rebuilt with build_cases.py --all. Sheet pastes are
  kept; fetched blocks sit beside them under the NOT YET TRIMMED banner.
  Files previously committed with CRLF are now LF.
- wayback-availability.json: Wayback snapshots exist for 73 of the 98
  failed URL slots; input for an archive-fetch pass.
- README-START-HERE.md: progress note and next step.

verify_all.py PASSES (1836 checks).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
This commit is contained in:
StellarCrow
2026-09-05 15:02:39 +02:00
co-authored by Claude Fable 5.1
parent ba7529ea50
commit 499fd61f79
785 changed files with 69932 additions and 16771 deletions
+15
View File
@@ -0,0 +1,15 @@
Mohammed Khubaib jailed over kidnapped teen ransom demand
A kidnapper who snatched a teenage boy from the street then demanded a ransom from his family has been jailed.
Mohammed Khubaib, 22, abducted the 14-year-old outside a Bradford takeaway in May and, with the help of others, bundled him into a car.
The boy's mother was blackmailed into paying £900 for his safe return, West Yorkshire Police said.
Khubaib earlier pleaded guilty to kidnap and blackmail and was jailed for four years at Bradford Crown Court.
Det Con Paul Maxwell said the boy was "vulnerable to criminal exploitation" and that Khubaib "tried to exploit him for money".
He added: "This was an incredibly distressing incident for both the victim and the victim's mother, but thankfully both were not injured during the incident and have been given help and support by the police and partner agencies."
Follow BBC Yorkshire on Facebook, Twitter and Instagram. Send your story ideas to yorkslincs.news@bbc.co.uk or send video here.
+16
View File
@@ -0,0 +1,16 @@
[
{
"case": 86,
"n": 1,
"field": "url",
"url": "https://www.bbc.com/news/uk-england-leeds-58929107",
"host": "bbc.com",
"fetched_at": "2026-09-05T12:53:49+00:00",
"verdict": "FETCHED",
"method": "raw",
"http": 200,
"chars": 1006,
"sha256": "bd5642c3a255f79eec800be60da33c88f48957d89f7a0accd5ffa2e94728e7fd",
"final_url": null
}
]