Scrape status
Refreshes every 20s · source of truth:
scrape_runs · showing 7 of 77 sources · filtered by failing · clear filter| Source | Status | When | Last run | Landed | Cadence | Warnings | |
|---|---|---|---|---|---|---|---|
| census-migration US Census county-to-county migration flows — needs CENSUS_API_KEY (free), 2020 vintage (last with true county detail, verified live), ~4M rows national scope. |
degraded | 22d ago (2026-10-24 11:06 EDT) | 1155275 rows · 0 new · 0 dropped | 1404318 | monthly | ⚠ 9 | |
| doordash-zip derives a ZIP for DoorDash stores (their pages publish none) from the street address (Census batch geocoder) then the coordinates' ZCTA; stored in zip_derived with its method; measured 96.2% equal to published ZIPs on 400 Uber Eats stores |
degraded | 15h ago (2026-11-15 18:09 EST) | 49998 rows · 49998 new · 0 dropped | 679464 | daily | ⚠ 1 | |
| mdm-briefing DOCS — Claude writes the MDM docs page's 'Next steps and suggestions' (dashboard /docs) once a week, after the master build: where the master stands, what changed since the last briefing, and grounded next steps with who should do them. Read-only tools (run history, the package's code, the docs, one SELECT at a time); changes nothing. Weekly and on Sonnet since 2026-11-13, to cut spend. Budget MDM_BRIEFING_MAX_USD (default $1). See mdm_briefing.py. |
degraded | 1d ago (2026-11-14 16:05 EST) | 0 rows · 0 new · 0 dropped | 4 | weekly | ⚠ 1 | |
| source-health MONITOR — a DERIVED build over scrape_runs: rules find every failed, degraded, resting, stale, never-run or shrunken source; Claude (read-only tools: run history, the package's code, one SELECT at a time) diagnoses each new or changed finding — cause, evidence, next action. Changes nothing. Budget SOURCE_HEALTH_MAX_USD (default $0.50) per run, on Sonnet. See source_health.py. |
degraded | 9h ago (2026-11-16 00:05 EST) | 6 rows · 2 new · 5 dropped | 31 | daily | ⚠ 2 | |
| tax-rates Bev-alc tax rate reference — encoded federal schedule + a curated state seed CSV, not a live crawl. |
degraded | 6d ago (2026-11-09 16:03 EST) | 171 rows · 0 new · 0 dropped | 171 | weekly | ⚠ 5 | |
| ubereats US store universe (26 public sitemaps, direct) + each store's address/ZIP/lat-lng/chain via getStoreV1 THROUGH THE ISP POOL (a bare Railway IP is reCAPTCHA-challenged after a modest volume — measured); alcohol-named stores first, UE_BATCH per run. Items not landed yet (no alcohol marker) |
degraded | 1d ago (2026-11-15 05:45 EST) | 9893 rows · 0 new · 0 dropped | 1048889 | daily due | ⚠ 3 | |
| winebow Winebow importer brand portfolio — server-rendered HTML, regex parse, no auth. |
failed | 6d ago (2026-11-10 05:02 EST) | 0 rows · 0 new · 0 dropped | 1399 | weekly | ⚠ 2 |