fix: retry failed group subpages, add rate-limit detection in scraper
- Detect Wikipedia plain-text rate-limit response ("You are making too many
requests") and wait 30s before retrying, rather than silently failing
- Increase inter-attempt delay from 3s to 15s per attempt
- Increase group subpage delay from 1.2s to 3s, year delay from 0.6s to 2s
- Re-scrape 1982, 1998, 2002, 2006 which had failed groups; all groups now
complete — e.g. 2002 now has 64 matches including Group E (Germany/Klose)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -54,6 +54,14 @@
|
||||
"New Zealand"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Group A",
|
||||
"teams": [
|
||||
"Poland",
|
||||
"Belgium",
|
||||
"Soviet Union"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "Group B",
|
||||
"teams": [
|
||||
|
||||
Reference in New Issue
Block a user