feat(sources): Wayback pass, automatic trimming, detention rule, ten cases re-sourced
Second and third rounds on the dossier pipeline, reviewed locally before commit. Retrieval - wayback_pass.py: retries every BLOCKED/DEAD/THIN/ERROR linked source through the Wayback Machine; blocks carry the snapshot date, archive URL and the live verdict. 143 slots retried: 93 fetched, 37 no snapshot, 12 script shells, 1 PDF. - fetch_sources.py: fetch_wayback() helper. - add_extra_sources.py: files staged extra_* finds (web search, not on the DB record) into the store with a provenance note. - Cases re-sourced by web search: 248 (Oslo: Document.no, Avisa Oslo, NRK; the linked Le Parisien piece is case 242 and is marked OFF-CASE), 258 Kharkiv, 260 Singapore, 358 Bangkok, 365 Las Vegas, 388 Verneuil-sur-Seine, 390 Zoersel, 430 Homestead. 341 of 364 cases now hold a fetched article; 11, 66 and 399 have no public text source (podcast, police video, direct victim report). Trimming - auto_trim.py: case-anchored furniture cut for UNTRIMMED blocks. Finds the body run that mentions the case, merges across subheadings and short furniture gaps, drops teasers, share bars, date/URL/caption lines and subscription pitches; refuses pages with no record term or almost no body. 302 blocks trimmed; 16 left on auto-trim-review.md (7 OFF-CASE suspects). Every cut is labelled AUTO-TRIMMED, UNREVIEWED in the dossier; the full extract stays in staging/. Decisions with anchors in trim-decisions-auto.json. - build_cases.py / verify_all.py: banners for auto-trimmed and archived blocks, staged-file check extended to Wayback blocks, flag legend under the record. Detention rule - README "Definitions": the DB field `kidnappings` is the DETENTION violence type (victim, guard, staff or relative held to force submission or execute the theft), distinct from the Kidnapping scenario (taken away and held). - detention-flag-review.md / detention-flag-corrections.json: 40 records reviewed with evidence; 30 set-to-1 proposals accepted by the owner on 2026-09-06 (listed in corrections-approved.md), 10 still open. - apply_detention_wording.py: "Violence Used" in the 70 reviewed summaries now names detention explicitly (62 of 70 labelled), supported by the summary's own text; idempotent; supersedes the batch scripts' wording. Worklist and docs - sofia-worklist.md: 248 decisions, detention rule item replacing the old "no abduction" item, fresh-search section for the textless cases. - README-START-HERE.md: progress notes, run commands, next steps. - Bug fixed in passing: Wayback blocks stored in-memory text with carriage returns; now stored as read back from disk. verify_all.py PASSES (2321 checks). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GZZENdTLzNGsbNy4DyF1yt
This commit is contained in:
StellarCrow
co-authored by
Claude Fable 5.1
Claude Fable 5.1
parent
499fd61f79
commit
439e83a4cb
@@ -26,6 +26,9 @@ DATABASE RECORD
|
||||
notes A drug addict lent her basement to an acquaintance who wanted to steal the young man's cryptocurrency wallet. They kidnapped him, a friend, and 2 girls. The 3 were freed the next day.
|
||||
has_processed_date 0
|
||||
created_at 2025-12-15 15:26:30
|
||||
(flag legend: 'kidnappings' is the DETENTION flag — 1 when the victim was held against
|
||||
their will at any point, tied, locked in, held at gunpoint or taken away; it is NOT the
|
||||
same as the 'Kidnapping' scenario, which means taken away and held. See README, Definitions.)
|
||||
|
||||
AI SUMMARY [generated at bulk import, NOT verified against sources]
|
||||
**Victim:** Kevin Mirshahi, 25-year-old cryptocurrency influencer from Montreal
|
||||
@@ -45,10 +48,6 @@ LINKED SOURCES
|
||||
[url_3] https://www.ctvnews.ca/montreal/article/police-arrest-2-suspects-in-killing-of-montreal-cryptocurrency-influencer/
|
||||
[url_4] https://www.cbc.ca/news/canada/montreal/kevin-mirshahi-arrests-1.7387589
|
||||
|
||||
OTHER LINKED SOURCES
|
||||
[THIN] url_3 ctvnews.ca — JS-rendered index page; only unrelated teaser headlines extracted (no article body). Case 379 is well-sourced by lapresse/globalnews/cbc, so this can be dropped.
|
||||
https://www.ctvnews.ca/montreal/article/police-arrest-2-suspects-in-killing-of-montreal-cryptocurrency-influencer/
|
||||
|
||||
------------------------------------------------------------------------------------------------
|
||||
FETCHED ARTICLE 1
|
||||
Source lapresse.ca
|
||||
@@ -166,7 +165,181 @@ Mirshahi, 25, disappeared after he and three other people in their 20s were kidn
|
||||
By August, investigators had concluded Mirshahi had been killed. Police arrested Joanie Lepage, 32, of Les Cèdres, 45 kilometres southwest of Montreal. Lepage was charged with first-degree murder, forcible confinement and accessory after the fact to murder on Aug. 22.
|
||||
|
||||
------------------------------------------------------------------------------------------------
|
||||
FETCHED ARTICLE 3
|
||||
FETCHED ARTICLE 3 [RAW EXTRACT — NOT YET TRIMMED]
|
||||
!! This is the whole page as extracted. It still contains site furniture,
|
||||
!! and may contain teaser headlines for UNRELATED cases. Trim on read-through.
|
||||
!! Do not quote from it without checking the passage belongs to this case.
|
||||
Source ctvnews.ca
|
||||
URL https://www.ctvnews.ca/montreal/article/police-arrest-2-suspects-in-killing-of-montreal-cryptocurrency-influencer/
|
||||
Field url_3
|
||||
Retrieved 2026-09-05T13:24:55+00:00 via Wayback Machine snapshot 20250610215715, deterministic extraction (HTTP None)
|
||||
Language unknown
|
||||
Kept paragraphs ALL 76 paragraphs, untrimmed
|
||||
Length 5,932 chars
|
||||
Integrity sha256 fc0f09ee6d29515b76e70f3d230341c05726bfced7767b893ba0ae2cf7a6664a
|
||||
Trimmed NOT TRIMMED. Full page extract; trim on read-through.
|
||||
Note live URL was THIN on the first pass; text comes from an archived copy dated 2025-06-10
|
||||
Archive http://web.archive.org/web/20250610215715/https://www.ctvnews.ca/montreal/article/police-arrest-2-suspects-in-killing-of-montreal-cryptocurrency-influencer/
|
||||
Caveat retrieved from a Wayback Machine snapshot, not the live page. The
|
||||
snapshot date is in the method line; the live URL was unreadable.
|
||||
------------------------------------------------------------------------------------------------
|
||||
|
||||
ORIGINAL (unknown, verbatim as retrieved)
|
||||
|
||||
Police arrest 2 suspects in killing of Montreal cryptocurrency influencer
|
||||
|
||||
Skip to main content
|
||||
|
||||
Police arrest 2 suspects in killing of Montreal cryptocurrency influencer
|
||||
|
||||
Published: November 19, 2024 at 2:03PM EST
|
||||
|
||||
Father and 14-year-old son charged after Orono mosque vandalized, antisemitic pamphlets placed at Bowmanville store
|
||||
|
||||
Victim identified in deadly Scarborough shooting
|
||||
|
||||
B.C. professor given prestigious award for tireless efforts protecting Borneo’s orangutans
|
||||
|
||||
Crown to consider charging RCMP officer who fatally shot suspected car thief in Revelstoke, B.C.
|
||||
|
||||
House ablaze in northwest Calgary; fire department calls in Springbank, RVC for help
|
||||
|
||||
Calgary firefighter injured in hit-and-run involving stolen car
|
||||
|
||||
Homeowners fed up, demand accountability after flood destroys Embrun homes
|
||||
|
||||
86 packages containing drugs seized at Canada Post location in Ottawa
|
||||
|
||||
Judge says Montreal billionaire Robert Miller too sick for trial on sex charges
|
||||
|
||||
Hobby shops say it will be game over for their businesses due to Quebec’s language law
|
||||
|
||||
Child luring charges laid against Grande Prairie man, RCMP seeking other victims
|
||||
|
||||
Edmonton Oilers look to turn Stanley Cup final page after emphatic Game 3 loss
|
||||
|
||||
Man convicted in Cape Breton hit and run granted day parole, mother of victim says
|
||||
|
||||
Halifax council votes down motion to pause construction of bike lane construction
|
||||
|
||||
Credit rating agency says Manitoba’s recent tax changes outweigh affordability offers
|
||||
|
||||
Protester who broke Lemay Forest court injunction fined $15K
|
||||
|
||||
Sask. ombudsman investigating provincial wildfire response, handling of evacuees
|
||||
|
||||
Sask. educational assistant sentenced to 28 months in prison for sexual exploitation
|
||||
|
||||
Alleged Canadian hacker unmasked after threatening cybersecurity researcher
|
||||
|
||||
‘We’re triaging’: Sask. Public Safety Agency says it’s aware of support backlog, working to rectify
|
||||
|
||||
Canadians believe they need $1 million to retire comfortably: report
|
||||
|
||||
St. Thomas apartment dwellers with no utilities, landlord wanted in U.S. for grandparent scam
|
||||
|
||||
Mayor’s lobbying suddenly yields $10M savings in 2026 budget from Development Charge exemptions
|
||||
|
||||
Barrie woman’s family desperate for answers 2 years after her mysterious disappearance
|
||||
|
||||
Ontario police seize $5.4 million in drugs in two massive trafficking investigations
|
||||
|
||||
Here’s how the city is closing off water access at Sandpoint Beach
|
||||
|
||||
Free shuttle service offered for Ford Fireworks
|
||||
|
||||
U.S. man sentenced for child luring in case involving Saanich, B.C., girl
|
||||
|
||||
Six-foot boa constrictor goes missing from beach in Tofino, B.C.
|
||||
|
||||
Satanic Temple accuses Iowa officials of religious discrimination
|
||||
|
||||
Baltimore jail staff failed to protect a deaf man killed by his cellmate, lawsuit says
|
||||
|
||||
Prime minister, MPs attend ceremony to raise Pride flag on Parliament Hill
|
||||
|
||||
Federal government slow to reduce, modernize its office space
|
||||
|
||||
Auditor general finds F-35 costs soar amid project delays, pilot shortages
|
||||
|
||||
Union says Canada Post won’t meet it ‘halfway’ on arbitration, prolonging deadlock
|
||||
|
||||
Live Nation to invest US$1 billion in U.S. music venues
|
||||
|
||||
Global economy on track for worst decade since 1960s, World Bank warns
|
||||
|
||||
NIH scientists publish declaration criticizing Trump’s deep cuts in public health research
|
||||
|
||||
Teen TikTok skin care craze may be harmful — and expensive, study finds
|
||||
|
||||
Obese patients may need custom doses of certain antibiotics, research suggests
|
||||
|
||||
Weinstein jury pores over accuser’s emails during deliberations
|
||||
|
||||
A family thought the sculpture on their piano was a ‘fake’ Rodin. Now it’s sold for nearly US$1 million
|
||||
|
||||
Judge denies Sean ‘Diddy’ Combs’ second request for mistrial, likens cross-examination to ‘Perry Mason moment’
|
||||
|
||||
Canada’s Andreescu cruises into second round of Libema Open with win over Garland
|
||||
|
||||
Edmonton Oilers superfans cross ocean, paint body in blue and orange to watch games
|
||||
|
||||
World’s tallest monk? Victor Wembanyama spotted at China’s Shaolin Temple with shaved head
|
||||
|
||||
‘It was a form of healing’: How travelling to every country helped this American cope with a mental health disorder
|
||||
|
||||
200-year-old condom decorated with erotic art goes on display in Amsterdam’s Rijksmuseum
|
||||
|
||||
At $3,000 a night, luxury farm resorts are the next glamorous getaway
|
||||
|
||||
Ottawa’s plan for climate change adaptation is falling short, report says
|
||||
|
||||
Nations call for ‘quieter’ ocean to help marine life
|
||||
|
||||
Nearly all of Canada will see a warmer than average summer, say forecasters
|
||||
|
||||
Astronomy show accidentally reveals unseen structure in our solar system
|
||||
|
||||
New AI minister says Canada won’t ‘over-index’ on AI regulation
|
||||
|
||||
How scammers are using AI to steal college financial aid
|
||||
|
||||
Mazda recall: Battery issue may deactivate airbags on 170,000 vehicles
|
||||
|
||||
Investors dump Tesla on bet Trump may lash out at Musk through his car company
|
||||
|
||||
Ford Explorer recall: 23,000 SUVs pulled over risk of detaching parts
|
||||
|
||||
Los Angeles on edge: Clashes between police, protesters amid immigration crackdown
|
||||
|
||||
These Toronto-area neighbourhoods are now in 'underbidding territory'
|
||||
|
||||
Western Canada wildfire smoke creates hazy skies in parts of North America
|
||||
|
||||
20 Next-Level Father's Day Gifts You Can Get On Amazon Canada
|
||||
|
||||
14 Alcohol-Free Beers, Spirits, And Pre-Mixed Mocktails That Taste Just As Good As The Real Deal
|
||||
|
||||
17 Baby Essentials for New Parents Under $100
|
||||
|
||||
The Absolute Best Spice Racks You Can Get In Canada Right Now
|
||||
|
||||
I Tested A Bird Feeder With A Camera — Here Are My Honest Opinions
|
||||
|
||||
18 Handy Amazon Canada Products That'll Help You Save Some Money
|
||||
|
||||
I Tried 4 Popular Non-Aerosol Dry Shampoos, And Here Are My Thoughts
|
||||
|
||||
The Absolute Best Sunscreen For Oily Skin You Can Get In Canada Right Now
|
||||
|
||||
If Skin Elasticity Is Your Main Concern, Here Are 11 Of The Best Firming Eye Creams You Can Get In Canada Right Now
|
||||
|
||||
20 Last-Minute Father's Day Gifts That'll Arrive On Time
|
||||
|
||||
The Shopping Trends team is independent of the journalists at CTV News. We may earn a commission when you use our links to shop. Read about us.
|
||||
|
||||
------------------------------------------------------------------------------------------------
|
||||
FETCHED ARTICLE 4
|
||||
Source cbc.ca
|
||||
URL https://www.cbc.ca/news/canada/montreal/kevin-mirshahi-arrests-1.7387589
|
||||
Field url_4
|
||||
|
||||
Reference in New Issue
Block a user