valknar/worldcup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: make squad sync opt-in to prevent OOM in scheduled runs

Browse Source 
The squads Wikipedia page for 48 teams is very large and was causing the
scheduled sync to be OOM-killed (exit 137) mid-run. Squad data doesn't
change during the tournament, so it doesn't need to run on every sync.
Pass --squads to include it when needed (e.g. initial setup).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
valknar
2026-07-05 09:32:23 +02:00
co-authored by Claude Sonnet 4.6
parent 02d977a866
commit a437affd1d
1 changed files with 19 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/sync.ts
+19 -17
View File
@@ -204,25 +204,27 @@ async function run() {
goalCount += goals.length
}
// Squads (fetch once; idempotent upsert so safe to re-run)
const squadHtml = await fetchWikiHtml('2026_FIFA_World_Cup_squads')
if (squadHtml) {
const squads = scrapeSquads(squadHtml)
for (const sq of squads) {
const teamId = await upsertTeam(sq.name)
for (const p of sq.players) {
const dob = p.date_of_birth ? p.date_of_birth.replace(/\s/g, '') : null
await db.execute(sql`
INSERT INTO squads (tournament_year, team_id, player_name, shirt_number, position, date_of_birth)
VALUES (2026, ${teamId}, ${p.name}, ${p.number ?? null}, ${p.pos ?? null}, ${dob})
ON CONFLICT (tournament_year, team_id, shirt_number) DO UPDATE SET
player_name = EXCLUDED.player_name,
position = EXCLUDED.position,
date_of_birth = EXCLUDED.date_of_birth
`)
// Squads — opt-in only (large page, causes OOM in scheduled runs)
if (process.argv.includes('--squads')) {
const squadHtml = await fetchWikiHtml('2026_FIFA_World_Cup_squads')
if (squadHtml) {
const squads = scrapeSquads(squadHtml)
for (const sq of squads) {
const teamId = await upsertTeam(sq.name)
for (const p of sq.players) {
const dob = p.date_of_birth ? p.date_of_birth.replace(/\s/g, '') : null
await db.execute(sql`
INSERT INTO squads (tournament_year, team_id, player_name, shirt_number, position, date_of_birth)
VALUES (2026, ${teamId}, ${p.name}, ${p.number ?? null}, ${p.pos ?? null}, ${dob})
ON CONFLICT (tournament_year, team_id, shirt_number) DO UPDATE SET
player_name = EXCLUDED.player_name,
position = EXCLUDED.position,
date_of_birth = EXCLUDED.date_of_birth
`)
}
}
console.log(` Squads: ${squads.length} teams`)
}
console.log(` Squads: ${squads.length} teams`)
}
// Tournament winner (once the final is played)