fix: retry failed group subpages, add rate-limit detection in scraper
- Detect Wikipedia plain-text rate-limit response ("You are making too many
requests") and wait 30s before retrying, rather than silently failing
- Increase inter-attempt delay from 3s to 15s per attempt
- Increase group subpage delay from 1.2s to 3s, year delay from 0.6s to 2s
- Re-scrape 1982, 1998, 2002, 2006 which had failed groups; all groups now
complete — e.g. 2002 now has 64 matches including Group E (Germany/Klose)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -77,17 +77,29 @@
|
||||
"city": "Ulsan"
|
||||
},
|
||||
{
|
||||
"name": "Kashima Soccer Stadium",
|
||||
"city": "Ibaraki"
|
||||
},
|
||||
{
|
||||
"name": "Wing Stadium",
|
||||
"city": "Kobe"
|
||||
"name": "Niigata Stadium",
|
||||
"city": "Niigata"
|
||||
},
|
||||
{
|
||||
"name": "Sapporo Dome",
|
||||
"city": "Sapporo"
|
||||
},
|
||||
{
|
||||
"name": "Kashima Soccer Stadium",
|
||||
"city": "Ibaraki"
|
||||
},
|
||||
{
|
||||
"name": "Ecopa Stadium",
|
||||
"city": "Shizuoka"
|
||||
},
|
||||
{
|
||||
"name": "International Stadium Yokohama",
|
||||
"city": "Yokohama"
|
||||
},
|
||||
{
|
||||
"name": "Wing Stadium",
|
||||
"city": "Kobe"
|
||||
},
|
||||
{
|
||||
"name": "Niigata Big Swan Stadium",
|
||||
"city": "Niigata"
|
||||
@@ -96,10 +108,6 @@
|
||||
"name": "Ōita Big Eye Stadium",
|
||||
"city": "Ōita"
|
||||
},
|
||||
{
|
||||
"name": "International Stadium Yokohama",
|
||||
"city": "Yokohama"
|
||||
},
|
||||
{
|
||||
"name": "Shizuoka Ecopa Stadium",
|
||||
"city": "Fukuroi, Shizuoka"
|
||||
|
||||
Reference in New Issue
Block a user