fix: retry failed group subpages, add rate-limit detection in scraper

- Detect Wikipedia plain-text rate-limit response ("You are making too many
  requests") and wait 30s before retrying, rather than silently failing
- Increase inter-attempt delay from 3s to 15s per attempt
- Increase group subpage delay from 1.2s to 3s, year delay from 0.6s to 2s
- Re-scrape 1982, 1998, 2002, 2006 which had failed groups; all groups now
  complete — e.g. 2002 now has 64 matches including Group E (Germany/Klose)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-15 18:44:54 +02:00
parent d37ebe201e
commit 7e4bf2d07c
18 changed files with 1193 additions and 16 deletions
+148
View File
@@ -1430,6 +1430,154 @@
],
"ground": "Gottlieb-Daimler-Stadion, Stuttgart"
},
{
"round": "Group stage",
"group": "Group G",
"date": "2006-06-13",
"time": "15:00",
"team1": "South Korea",
"team2": "Togo",
"score": {
"ft": [
2,
1
]
},
"goals1": [
{
"name": "Lee Chun-soo",
"minute": 54
},
{
"name": "Ahn Jung-hwan",
"minute": 72
}
],
"goals2": [
{
"name": "Mohamed Kader",
"minute": 31
}
],
"ground": "Waldstadion, Frankfurt"
},
{
"round": "Group stage",
"group": "Group G",
"date": "2006-06-13",
"time": "18:00",
"team1": "France",
"team2": "Switzerland",
"score": {
"ft": [
0,
0
]
},
"ground": "Gottlieb-Daimler-Stadion, Stuttgart"
},
{
"round": "Group stage",
"group": "Group G",
"date": "2006-06-18",
"time": "21:00",
"team1": "France",
"team2": "South Korea",
"score": {
"ft": [
1,
1
]
},
"goals1": [
{
"name": "Thierry Henry",
"minute": 9
}
],
"goals2": [
{
"name": "Park Ji-sung",
"minute": 81
}
],
"ground": "Zentralstadion, Leipzig"
},
{
"round": "Group stage",
"group": "Group G",
"date": "2006-06-19",
"time": "15:00",
"team1": "Togo",
"team2": "Switzerland",
"score": {
"ft": [
0,
2
]
},
"goals2": [
{
"name": "Alexander Frei",
"minute": 16
},
{
"name": "Tranquillo Barnetta",
"minute": 88
}
],
"ground": "Westfalenstadion, Dortmund"
},
{
"round": "Group stage",
"group": "Group G",
"date": "2006-06-23",
"time": "21:00",
"team1": "Togo",
"team2": "France",
"score": {
"ft": [
0,
2
]
},
"goals2": [
{
"name": "Patrick Vieira",
"minute": 55
},
{
"name": "Thierry Henry",
"minute": 61
}
],
"ground": "RheinEnergieStadion, Cologne"
},
{
"round": "Group stage",
"group": "Group G",
"date": "2006-06-23",
"time": "21:00",
"team1": "Switzerland",
"team2": "South Korea",
"score": {
"ft": [
2,
0
]
},
"goals1": [
{
"name": "Philippe Senderos",
"minute": 23
},
{
"name": "Alexander Frei",
"minute": 77
}
],
"ground": "Niedersachsenstadion, Hanover"
},
{
"round": "Group stage",
"group": "Group H",