valknar/worldcup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: make squad sync opt-in to prevent OOM in scheduled runs

Browse Source 
The squads Wikipedia page for 48 teams is very large and was causing the
scheduled sync to be OOM-killed (exit 137) mid-run. Squad data doesn't
change during the tournament, so it doesn't need to run on every sync.
Pass --squads to include it when needed (e.g. initial setup).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
valknar
2026-07-05 09:32:23 +02:00
co-authored by Claude Sonnet 4.6
parent 02d977a866
commit a437affd1d
1 changed files with 19 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/sync.ts
+19 -17
View File
@@ -204,25 +204,27 @@ async function run() {
goalCount += goals.length goalCount += goals.length
} }
// Squads (fetch once; idempotent upsert so safe to re-run) // Squads — opt-in only (large page, causes OOM in scheduled runs)
const squadHtml = await fetchWikiHtml('2026_FIFA_World_Cup_squads') if (process.argv.includes('--squads')) {
if (squadHtml) { const squadHtml = await fetchWikiHtml('2026_FIFA_World_Cup_squads')
const squads = scrapeSquads(squadHtml) if (squadHtml) {
for (const sq of squads) { const squads = scrapeSquads(squadHtml)
const teamId = await upsertTeam(sq.name) for (const sq of squads) {
for (const p of sq.players) { const teamId = await upsertTeam(sq.name)
const dob = p.date_of_birth ? p.date_of_birth.replace(/\s/g, '') : null for (const p of sq.players) {
await db.execute(sql` const dob = p.date_of_birth ? p.date_of_birth.replace(/\s/g, '') : null
INSERT INTO squads (tournament_year, team_id, player_name, shirt_number, position, date_of_birth) await db.execute(sql`
VALUES (2026, ${teamId}, ${p.name}, ${p.number ?? null}, ${p.pos ?? null}, ${dob}) INSERT INTO squads (tournament_year, team_id, player_name, shirt_number, position, date_of_birth)
ON CONFLICT (tournament_year, team_id, shirt_number) DO UPDATE SET VALUES (2026, ${teamId}, ${p.name}, ${p.number ?? null}, ${p.pos ?? null}, ${dob})
player_name = EXCLUDED.player_name, ON CONFLICT (tournament_year, team_id, shirt_number) DO UPDATE SET
position = EXCLUDED.position, player_name = EXCLUDED.player_name,
date_of_birth = EXCLUDED.date_of_birth position = EXCLUDED.position,
`) date_of_birth = EXCLUDED.date_of_birth
`)
}
} }
console.log(` Squads: ${squads.length} teams`)
} }
console.log(` Squads: ${squads.length} teams`)
} }
// Tournament winner (once the final is played) // Tournament winner (once the final is played)