fix: retry failed group subpages, add rate-limit detection in scraper

- Detect Wikipedia plain-text rate-limit response ("You are making too many
  requests") and wait 30s before retrying, rather than silently failing
- Increase inter-attempt delay from 3s to 15s per attempt
- Increase group subpage delay from 1.2s to 3s, year delay from 0.6s to 2s
- Re-scrape 1982, 1998, 2002, 2006 which had failed groups; all groups now
  complete — e.g. 2002 now has 64 matches including Group E (Germany/Klose)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-15 18:44:54 +02:00
parent d37ebe201e
commit 7e4bf2d07c
18 changed files with 1193 additions and 16 deletions
+18 -10
View File
@@ -77,17 +77,29 @@
"city": "Ulsan"
},
{
"name": "Kashima Soccer Stadium",
"city": "Ibaraki"
},
{
"name": "Wing Stadium",
"city": "Kobe"
"name": "Niigata Stadium",
"city": "Niigata"
},
{
"name": "Sapporo Dome",
"city": "Sapporo"
},
{
"name": "Kashima Soccer Stadium",
"city": "Ibaraki"
},
{
"name": "Ecopa Stadium",
"city": "Shizuoka"
},
{
"name": "International Stadium Yokohama",
"city": "Yokohama"
},
{
"name": "Wing Stadium",
"city": "Kobe"
},
{
"name": "Niigata Big Swan Stadium",
"city": "Niigata"
@@ -96,10 +108,6 @@
"name": "Ōita Big Eye Stadium",
"city": "Ōita"
},
{
"name": "International Stadium Yokohama",
"city": "Yokohama"
},
{
"name": "Shizuoka Ecopa Stadium",
"city": "Fukuroi, Shizuoka"