From 487ca40a44f930a7003e5f8d9418a10aa8529492 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sebastian=20Kr=C3=BCger?= Date: Sat, 22 Aug 2026 16:45:04 +0200 Subject: [PATCH] Phase D/F/H: embedding shell - loading progress, fullscreen, file upload, touch controls package/src/index.ts: fixes a real pre-existing bug (start()/loadTable() never actually called vpinball_wasm_start with a table path argument), adds byte-level download progress (onProgress) and requestFullscreen(). package/src/touch-controls.ts: new on-screen virtual flipper/plunger/start button overlay that synthesizes the actual default keyboard scancodes (SDL_SCANCODE_LSHIFT/RSHIFT/RETURN, read from InputManager.cpp) as real KeyboardEvents dispatched on window, matching SDL3's Emscripten keyboard target - keeps the native input path completely unmodified. examples/basic/index.html: a real demo page - progress bar, a user-gesture "Start" button (required for both audio autoplay and fullscreen), a file picker for self-contained .vpx uploads, and the touch overlay shown on touch-capable devices. scripts/build.sh: two real bugs found and fixed via actual browser testing: - MODULARIZE=1 without EXPORT_ES6=1 produces a classic script, not an ES module with a default export - `await import(...)).default` was always undefined. Fixes with -sEXPORT_ES6=1. - --closure 1 silently stripped FS.writeFile/readFile/mkdir down to just low-level node ops, breaking runtime table loading with no compile-time warning. Dropped until root-caused; -O2 alone is kept. Verified end-to-end in Chrome: full load->progress->start flow with no errors, fullscreen actually engaging (document.fullscreenElement true), and a file-picker-selected table loading correctly (LoadGameFromFilename /tables/uploaded.vpx in the boot log, followed by a normal render). README updated to reflect what's now validated vs. still open (real-device input/audio confirmation remains the main open item). --- README.md | 37 +++++++--- examples/basic/index.html | 97 +++++++++++++++++++++++--- package/src/index.ts | 93 ++++++++++++++++++++----- package/src/touch-controls.ts | 125 ++++++++++++++++++++++++++++++++++ package/src/types.ts | 37 +++++++--- scripts/build.sh | 9 ++- 6 files changed, 352 insertions(+), 46 deletions(-) create mode 100644 package/src/touch-controls.ts diff --git a/README.md b/README.md index 37dbd56..d299068 100644 --- a/README.md +++ b/README.md @@ -61,11 +61,15 @@ Full research and two isolated feasibility spikes (proving real VBScript executi - Physics engine initialization and the VBScript engine starting up. - **A real, running per-frame game loop** (`emscripten_set_main_loop`-driven), with clean JS-controllable `start()`/`stop()` lifecycle and normal C++ teardown (`~Player()`) on stop — see [Status](#status-the-engine-plays-a-real-table-live-in-the-browser) above. - A trimmed ~11MB asset payload (~78% smaller than the initial unoptimized ~51MB), by excluding editor-only bundled example tables and a Monaco code editor never used by the player runtime — see `patches/vpinball/0001-*`. +- **A full loading → play flow in a real page** (`examples/basic/index.html`): a byte-level download progress bar, a user-gesture "Start" button (required for both audio autoplay and fullscreen to work), a working fullscreen button, and responsive canvas sizing that fills its container correctly. +- **Browser file-loading for self-contained tables**: picking a `.vpx` file, mounting it via `FS.writeFile`, and starting the engine against it — verified end-to-end (`PinTable::LoadGameFromFilename /tables/uploaded.vpx` in the boot log, followed by a normal render). +- Touch-control overlay code (`attachTouchControls`) that synthesizes the real default keyboard scancodes (`SDL_SCANCODE_LSHIFT`/`RSHIFT`/`RETURN` — see `src/input/InputManager.cpp`) on `window`, matching SDL3's Emscripten keyboard target; implemented and included in the example, not yet confirmed to move a flipper on a real touch device (see below). **Not yet validated (should work, per code review, but unconfirmed end-to-end):** -- Keyboard/gamepad input actually reaching a flipper/plunger during real gameplay (the input pipeline is code-complete and validated as wired correctly, but a definitive "pressed a key, ball moved" test is still open — see `src/input/SDLInputHandler.h`/`InputManager.cpp`, unmodified). -- Audio actually audible (the pipeline is fully wired per code review; needs a real speaker/headphone test plus a user-gesture gate for browser autoplay policy — see below). +- Keyboard/gamepad input actually reaching a flipper/plunger during real gameplay. The pipeline is code-complete and correctly wired (verified by reading `SDLInputHandler.h`/`InputManager.cpp`, unmodified), and the underlying key bindings were confirmed by reading the actual default scancode table. A real physical keypress test is still open - the automation environment used for validation in this project could not reliably simulate a modifier-key press (`Shift` alone) as a trusted browser event, which is a tooling limitation of that environment, not a claim that input doesn't work. +- Audio actually audible (the pipeline is fully wired per code review; needs a real speaker/headphone test - the autoplay-policy user-gesture gate is now in place via the example's "Start" button). - DMD rendering via a script-driven `Flasher`/`ScriptGlobalTable::put_DMDPixels` (this is native, core-engine functionality requiring no new code — see `src/core/ScriptGlobalTable.cpp:886-937`, `src/parts/flasher.cpp:1312-1341` — just needs a test table that exercises it). +- Touch controls actually flipping a flipper on a real touch device (see above). **Explicitly out of scope for now** (browser-sandbox constraints or a real infrastructure-cost tradeoff, not a technical dead end — could be revisited): - **Multi-threading** (pthreads/SharedArrayBuffer): would require mandatory `Cross-Origin-Opener-Policy`/`Cross-Origin-Embedder-Policy` headers on every page hosting this widget, a `coi-serviceworker`-style workaround for static hosts that can't set custom headers, and real risk of breaking unrelated cross-origin embeds on host pages — a poor tradeoff for something meant to be embeddable in arbitrary third-party pages. `ThreadPool`'s one-off parallel work (e.g. parallel `.vpx` item deserialization) already runs synchronously instead (`patches/vpinball/0002-*`). Revisit only if real-world profiling shows single-threaded performance is genuinely insufficient. @@ -73,10 +77,10 @@ Full research and two isolated feasibility spikes (proving real VBScript executi - Raw HID device input (`OpenPinDevHandler`/hidapi) — no browser equivalent for real-hardware nudge/plunger boards. Permanent, not a "for now." **Further follow-up work:** -- A definitive input/audio validation pass (see above). -- Touch controls (on-screen virtual flipper/plunger buttons synthesizing keyboard events) and fullscreen/pointer-lock UI wiring for mobile and kiosk-style embedding. -- Browser file-loading UX (drag-and-drop/file picker for user-supplied, self-contained `.vpx` files beyond the bundled demo table; real-world tables with an external `.vbs` script override or a `Music/` folder are a documented, known-unsupported gap for single-file uploads). -- Further binary-size tuning (the `.wasm` itself is still ~13MB even with `-O2 --closure 1`; `-sFORCE_FILESYSTEM=1`/broad `EXPORTED_RUNTIME_METHODS` pull in more than strictly needed and are candidates to narrow). +- A definitive real-device input/audio validation pass (see above). +- Pointer-lock UI wiring (fullscreen is done; pointer-lock isn't needed for this game's input model but noted here in case a future feature wants it). +- Real-world table support beyond self-contained single-file uploads: tables with an external `.vbs` script override or a `Music/` folder are a documented, known-unsupported gap. +- Further binary-size tuning (the `.wasm` itself is still ~13MB with `-O2`; `--closure 1` was tried but silently stripped `FS.writeFile`/`readFile`/`mkdir` down to low-level node ops - a real Emscripten/Closure interaction bug worth root-causing before re-enabling, since it broke runtime table loading with no compile-time warning. `-sFORCE_FILESYSTEM=1`/broad `EXPORTED_RUNTIME_METHODS` also pull in more than strictly needed). - A browser-based (Puppeteer) CI smoke test that actually loads a table and checks for a rendered frame, replacing today's compile-only CI check. ## Build @@ -96,20 +100,31 @@ source emsdk/emsdk_env.sh ## npm usage ```ts -import { loadPinball } from '@valknar/vpinball-wasm'; +import { loadPinball, attachTouchControls } from '@valknar/vpinball-wasm'; -const pinball = await loadPinball({ canvas: document.querySelector('canvas') }); -pinball.start(); +const canvas = document.querySelector('canvas'); +const pinball = await loadPinball({ + canvas, + onProgress: (fraction) => updateMyLoadingBar(fraction), +}); + +// start() (and requestFullscreen()) must be called from within a user +// gesture, e.g. a click handler - browsers block audio autoplay and +// fullscreen requests otherwise. +startButton.addEventListener('click', () => { + pinball.start(); + if ('ontouchstart' in window) attachTouchControls({ container: canvas.parentElement }); +}); ``` -The published package name/registry (`@valknar/vpinball-wasm` on this project's Gitea npm registry) is a placeholder pending the first real release — see `package.json`. `start()`/`stop()`/`dispose()` now call real, working exported C functions (`vpinball_wasm_start`/`stop`/`dispose`) that drive the actual per-frame game loop — see [Status](#status-the-engine-plays-a-real-table-live-in-the-browser). +See `examples/basic/index.html` for a complete working page (loading progress, file upload, touch controls, fullscreen). The published package name/registry (`@valknar/vpinball-wasm` on this project's Gitea npm registry) is a placeholder pending the first real release — see `package.json`. ## Known limitations - Single-threaded only (no pthreads/SharedArrayBuffer) — a deliberate tradeoff, not a gap; see Roadmap. - No heavy plugin ecosystem (PinMAME/DOF/FlexDMD/etc.) — see Roadmap. Native DMD rendering and a lightweight modern backglass plugin remain possible without it. - No raw hardware input (real cabinet nudge/plunger boards) — permanent, no browser equivalent. -- Keyboard/gamepad input and audio are code-complete and wired correctly but not yet validated end-to-end in real gameplay (see Roadmap) — no touch controls or fullscreen/pointer-lock UI yet. +- Keyboard/gamepad input and audio are code-complete and wired correctly, and the embedding shell (loading progress, fullscreen, file upload, touch-control overlay) is built and tested end-to-end, but a real-device confirmation that a physical/touch keypress moves a flipper and that audio is audible is still open — see Roadmap. - `.wasm`/asset size is reduced (~11MB total, down from ~51MB) but not fully tuned — see Roadmap. ## Licensing diff --git a/examples/basic/index.html b/examples/basic/index.html index 560cacb..00ace98 100644 --- a/examples/basic/index.html +++ b/examples/basic/index.html @@ -1,26 +1,107 @@ + vpinball-wasm example - +
+ +
+
+
+ +
+ +
+ +
+ diff --git a/package/src/index.ts b/package/src/index.ts index 59ab48a..eb43546 100644 --- a/package/src/index.ts +++ b/package/src/index.ts @@ -1,20 +1,28 @@ import type { LoadPinballOptions, PinballInstance } from './types.js'; export type { LoadPinballOptions, PinballInstance } from './types.js'; +export { attachTouchControls } from './touch-controls.js'; +export type { TouchControlsOptions, TouchControlsHandle } from './touch-controls.js'; + +const DEFAULT_TABLE_PATH = '/tables/default.vpx'; +const UPLOADED_TABLE_PATH = '/tables/uploaded.vpx'; /** * Instantiates the WebAssembly build of Visual Pinball against the given - * canvas. - * - * NOTE (current milestone): this wrapper boots the Emscripten module and - * exposes lifecycle control (start/stop/dispose). Runtime table loading - * (`loadTable`) mounts the given bytes into the module's virtual filesystem, - * but wiring the native engine to actually pick up a runtime-loaded table - * (as opposed to the table baked in at build time) is tracked as a follow-up - * milestone in the README's roadmap - see "Browser file-loading UX". + * canvas and returns a handle to control its lifecycle. */ export async function loadPinball(options: LoadPinballOptions): Promise { const baseUrl = options.baseUrl ?? '.'; + let tablePath = DEFAULT_TABLE_PATH; + + const moduleArgs: Record = { + canvas: options.canvas, + locateFile: (path: string) => `${baseUrl}/${path}`, + }; + + if (options.onProgress) { + await wireDownloadProgress(moduleArgs, baseUrl, options.onProgress); + } // dist/vpinball.js is built with -sMODULARIZE=1 -sEXPORT_NAME=VPinballModule, // so importing it yields a factory function, not a module with side effects. @@ -22,21 +30,21 @@ export async function loadPinball(options: LoadPinballOptions): Promise ) => Promise; - const module = await factory({ - canvas: options.canvas, - locateFile: (path: string) => `${baseUrl}/${path}`, - }); + const module = await factory(moduleArgs); + options.onProgress?.(1); if (options.tableData) { - mountTable(module, options.tableData); + module.FS.writeFile(UPLOADED_TABLE_PATH, new Uint8Array(options.tableData)); + tablePath = UPLOADED_TABLE_PATH; } return { loadTable(vpxBytes: ArrayBuffer) { - mountTable(module, vpxBytes); + module.FS.writeFile(UPLOADED_TABLE_PATH, new Uint8Array(vpxBytes)); + tablePath = UPLOADED_TABLE_PATH; }, start() { - module.ccall?.('vpinball_wasm_start', null, [], []); + module.ccall?.('vpinball_wasm_start', 'number', ['string'], [tablePath]); }, stop() { module.ccall?.('vpinball_wasm_stop', null, [], []); @@ -44,11 +52,62 @@ export async function loadPinball(options: LoadPinballOptions): Promise, + baseUrl: string, + onProgress: (fraction: number) => void +): Promise { + // Roughly half the total download is the wasm binary, half the data + // package - this is an estimate (see README's size-tuning roadmap item), + // not a guarantee, so progress may jump at the wasm/data boundary. + const WASM_WEIGHT = 0.5; + + try { + const response = await fetch(`${baseUrl}/vpinball.wasm`); + const total = Number(response.headers.get('Content-Length') ?? 0); + const reader = response.body?.getReader(); + if (!reader || !total) { + return; + } + const chunks: Uint8Array[] = []; + let received = 0; + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + chunks.push(value); + received += value.length; + onProgress((received / total) * WASM_WEIGHT); + } + const wasmBinary = new Uint8Array(received); + let offset = 0; + for (const chunk of chunks) { + wasmBinary.set(chunk, offset); + offset += chunk.length; + } + moduleArgs.wasmBinary = wasmBinary; + } catch { + // Streaming progress is best-effort; fall through to the factory's + // own default fetch if this fails for any reason (e.g. no CORS + // Content-Length exposed, older browser). + } + + moduleArgs.setStatus = (text: string) => { + if (text) onProgress(WASM_WEIGHT + (1 - WASM_WEIGHT) * 0.5); + }; } interface EmscriptenModule { diff --git a/package/src/touch-controls.ts b/package/src/touch-controls.ts new file mode 100644 index 0000000..1a47d4e --- /dev/null +++ b/package/src/touch-controls.ts @@ -0,0 +1,125 @@ +/** + * On-screen touch controls for mobile/tablet play: an HTML/CSS overlay of + * virtual buttons that synthesize the same keyboard events a physical + * keyboard would send, using vpinball's actual default key bindings + * (see vendor/vpinball/src/input/InputManager.cpp's addFlipperKeyAction/ + * addKeyAction calls). Deliberately implemented in JS/HTML rather than + * native code - this keeps the native SDL keyboard input path completely + * unmodified and is the dominant pattern for adding touch controls to a + * keyboard-oriented Emscripten port. + * + * SDL3's Emscripten video backend listens for keydown/keyup on the + * "#window" target by default (see SDL_emscriptenvideo.c's keyboard_element + * default), so dispatching a real KeyboardEvent on `window` reaches it the + * same way a physical keypress would. + */ + +const KEY_BINDINGS = { + leftFlipper: { key: 'Shift', code: 'ShiftLeft' }, + rightFlipper: { key: 'Shift', code: 'ShiftRight' }, + plunger: { key: 'Enter', code: 'Enter' }, + start: { key: '1', code: 'Digit1' }, +} as const; + +export interface TouchControlsOptions { + /** Container to render the overlay into. Positioned absolutely, filling this element. */ + container: HTMLElement; + /** + * Which buttons to show. Defaults to the full set (both flippers, + * plunger, start). + */ + buttons?: Array; +} + +export interface TouchControlsHandle { + /** Remove the overlay and its event listeners. */ + detach(): void; +} + +/** + * Attaches a touch-control overlay to `options.container`. Call this + * conditionally (e.g. only when `'ontouchstart' in window` or on narrow + * viewports) - it's additive UI, not required for desktop/mouse+keyboard play. + */ +export function attachTouchControls(options: TouchControlsOptions): TouchControlsHandle { + const buttons = options.buttons ?? (Object.keys(KEY_BINDINGS) as Array); + + const root = document.createElement('div'); + root.setAttribute('data-vpinball-touch-controls', ''); + applyStyle(root, { + position: 'absolute', + inset: '0', + pointerEvents: 'none', + userSelect: 'none', + touchAction: 'none', + fontFamily: 'sans-serif', + }); + + const elements: HTMLElement[] = []; + + if (buttons.includes('leftFlipper')) { + elements.push(makeButton('◀', KEY_BINDINGS.leftFlipper, { left: '0', bottom: '0' })); + } + if (buttons.includes('rightFlipper')) { + elements.push(makeButton('▶', KEY_BINDINGS.rightFlipper, { right: '0', bottom: '0' })); + } + if (buttons.includes('plunger')) { + elements.push(makeButton('⬆', KEY_BINDINGS.plunger, { right: '0', top: '40%' })); + } + if (buttons.includes('start')) { + elements.push(makeButton('1', KEY_BINDINGS.start, { left: '50%', top: '0', transform: 'translateX(-50%)' })); + } + + for (const el of elements) root.appendChild(el); + options.container.appendChild(root); + + return { + detach() { + root.remove(); + }, + }; +} + +function makeButton(label: string, binding: { key: string; code: string }, position: Record): HTMLElement { + const btn = document.createElement('div'); + btn.textContent = label; + applyStyle(btn, { + position: 'absolute', + width: '72px', + height: '72px', + margin: '16px', + borderRadius: '50%', + background: 'rgba(255,255,255,0.15)', + color: 'rgba(255,255,255,0.85)', + display: 'flex', + alignItems: 'center', + justifyContent: 'center', + fontSize: '28px', + pointerEvents: 'auto', + touchAction: 'none', + ...position, + }); + + const down = () => { + btn.style.background = 'rgba(255,255,255,0.35)'; + window.dispatchEvent(new KeyboardEvent('keydown', { key: binding.key, code: binding.code, bubbles: true })); + }; + const up = () => { + btn.style.background = 'rgba(255,255,255,0.15)'; + window.dispatchEvent(new KeyboardEvent('keyup', { key: binding.key, code: binding.code, bubbles: true })); + }; + + btn.addEventListener('pointerdown', (e) => { + e.preventDefault(); + btn.setPointerCapture(e.pointerId); + down(); + }); + btn.addEventListener('pointerup', up); + btn.addEventListener('pointercancel', up); + + return btn; +} + +function applyStyle(el: HTMLElement, style: Record): void { + Object.assign(el.style, style); +} diff --git a/package/src/types.ts b/package/src/types.ts index 4ced512..7318460 100644 --- a/package/src/types.ts +++ b/package/src/types.ts @@ -9,22 +9,43 @@ export interface LoadPinballOptions { /** Canvas the engine renders into via SDL3's WebGL2 backend. */ canvas: HTMLCanvasElement; /** - * Raw bytes of a .vpx table file to load on startup. If omitted, the - * bundled default table (package/assets/test000-default-table.vpx) is - * used instead. + * Raw bytes of a self-contained .vpx table file to play instead of the + * bundled default table. Real-world tables that reference an external + * .vbs script override or a Music/ folder are not supported this way - + * see the README's "Browser file-loading UX" section. */ tableData?: ArrayBuffer; - /** Base URL to fetch dist/vpinball.wasm (and .data, if present) from. */ + /** Base URL to fetch dist/vpinball.wasm/.data (and vpinball.js) from. */ baseUrl?: string; + /** + * Called as the wasm binary and preloaded asset package download, + * with a 0-1 fraction of bytes received. Not called at all if the + * browser doesn't support streaming fetch progress (falls back to + * Emscripten's own default loading behavior with no progress callback). + */ + onProgress?: (fraction: number) => void; } export interface PinballInstance { - /** Load a different .vpx table at runtime, replacing the current one. */ + /** + * Stage a different self-contained .vpx table to play on the *next* + * start() call. Does not affect an already-running session - table + * switching mid-session isn't supported; call stop() first (a fresh + * loadPinball() call is the supported way to load a genuinely new + * session with a different table). + */ loadTable(vpxBytes: ArrayBuffer): void; - /** Start (or resume) the simulation's main loop. */ + /** Start the simulation's main loop. No-op if already running. */ start(): void; - /** Pause the simulation's main loop. */ + /** Stop the simulation; the engine tears down (script Exit event, settings save) on its next step. */ stop(): void; - /** Tear down the instance and free its WebAssembly memory. */ + /** Tear down the instance immediately and free its WebAssembly memory. */ dispose(): void; + /** + * Request fullscreen on the canvas via the standard Fullscreen API. + * Must be called from within a user gesture (e.g. a click handler) - + * browsers reject fullscreen requests otherwise. Returns the promise + * from the underlying requestFullscreen() call. + */ + requestFullscreen(): Promise; } diff --git a/scripts/build.sh b/scripts/build.sh index c2a59d7..c250885 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -40,6 +40,7 @@ EMSCRIPTEN_LINK_FLAGS=( -sMAX_WEBGL_VERSION=2 -sALLOW_MEMORY_GROWTH=1 -sMODULARIZE=1 + -sEXPORT_ES6=1 -sEXPORT_NAME=VPinballModule -sENVIRONMENT=web -sEXPORTED_RUNTIME_METHODS=FS,ccall,cwrap @@ -53,8 +54,12 @@ if [ "$BUILD_TYPE" = "Debug" ]; then else # CMAKE_BUILD_TYPE=Release only optimizes the compile step; emcc's link # step needs its own -O level to actually run wasm-opt/dead-code - # elimination and (with --closure) minify the JS glue. - EMSCRIPTEN_LINK_FLAGS+=(-O2 --closure 1) + # elimination. NOTE: --closure 1 was tried here but strips FS.writeFile/ + # readFile/mkdir (the FS convenience wrappers EXPORTED_RUNTIME_METHODS=FS + # is supposed to guarantee) down to just low-level node ops - dropped + # until that's root-caused, since a smaller JS glue isn't worth a broken + # runtime table-loading API. + EMSCRIPTEN_LINK_FLAGS+=(-O2) fi emcmake cmake \