WVP — WASM Video Player

A browser-based video player where demuxing runs in C++ compiled to WebAssembly via FFmpeg, and decoding is handled by WebCodecs (hardware) with FFmpeg software as fallback.

Features

• WebCodecs decode — H.264, HEVC, VP9, AV1, VP8 via browser hardware decoders
• FFmpeg software fallback — decodes anything WebCodecs can't handle
• Audio — WebCodecs AudioDecoder + Web Audio API scheduler; works in both decoder paths
• WebGL rendering — YUV420P planes uploaded directly; BT.709 conversion in the GPU shader
• File streaming — local files are read on demand via file.slice().arrayBuffer(); no full copy into WASM heap
• DASH support — MPD manifest parsing, SegmentTemplate (duration + timeline) and SegmentList
• Rotation — reads AV_PKT_DATA_DISPLAYMATRIX side data; corrects mobile video orientation via UV transform
• Decoder preference — Auto / WebCodecs / FFmpeg toggle; persisted in localStorage
• Seek — demuxer seek + WebCodecs decoder reset, or avformat_seek_file + codec flush
• Keyboard shortcuts — Space (play/pause), ←/→ (±5 s), M (mute)
• Fullscreen

Architecture

Browser (JS)                          WASM (C++)
────────────────────────────────      ────────────────────────────────────
File picker                           FileAVIO
  _regFile(file) → key                 js_file_read (EM_ASYNC_JS)
  wvp_open_demux_file(key)             js_file_size (EM_JS)
                                       avio_alloc_context → AVIOContext
URL / DASH fetch                      setup_format_ctx
  _pushBuffer → wvp_push_data          avformat_open_input
  wvp_open_demux()                     avformat_find_stream_info

WebCodecs path (preferred)            Demux-only path
  wvp_read_packet() → pkt data         av_read_frame
  VideoDecoder.decode(chunk)           returns 0=video / 1=audio / −1=EOF
  AudioDecoder.decode(chunk)
  VideoFrame → WebGL renderVideoFrame  FFmpeg fallback path
                                        wvp_open_codec → avcodec_open2
WebGL renderer (WGLRenderer)            wvp_decode_frame → 1=video frame
  LUMINANCE textures (Y/U/V planes)                      → 2=audio packet
  BT.709 fragment shader                sws_scale → YUV420P buffer
  UV transform matrix for rotation      wvp_get_y/u/v_plane → pointers

AudioScheduler                        Rotation
  AudioContext + GainNode               av_display_rotation_get
  schedule(AudioData)                   wvp_get_video_rotation → 0/90/180/270

Building

Prerequisites

• Emscripten (Homebrew: brew install emscripten) — tested with 5.x
• CMake ≥ 3.20

1 — Build FFmpeg for WebAssembly (once)

./build_ffmpeg.sh

Clones FFmpeg, cross-compiles with emconfigure/emmake, installs static libraries to third_party/ffmpeg-wasm/. Enabled codecs: H.264, HEVC, VP8, VP9, AV1, MPEG-4, AAC, MP3, Opus, Vorbis.

2 — Build the WASM module

# Release (optimised, LTO)
cmake -B build/release -DCMAKE_BUILD_TYPE=Release
cmake --build build/release

# Debug (DWARF symbols, Emscripten assertions)
cmake -B build/debug -DCMAKE_BUILD_TYPE=Debug
cmake --build build/debug

Outputs web/wvp.js and web/wvp.wasm.

3 — Serve

python3 -m http.server 8080 --directory web/
# open http://localhost:8080

Debugging C++ in Chrome

1. Build debug: cmake --build build/debug
2. Install the C/C++ DevTools Support (DWARF) extension
3. Serve from the project root:

   python3 -m http.server 8080
   # open http://localhost:8080/web/index.html

4. DevTools → Sources → decoder.cpp to set breakpoints

Project Structure

WVP/
├── web/
│   ├── index.html          UI — source picker, canvas, controls, decoder toggle
│   ├── style.css           Dark theme
│   ├── player.js           WVPPlayer, WebCodecsDecoder, WGLRenderer,
│   │                       AudioScheduler, DASHLoader, WASM bridge
│   ├── favicon.ico
│   ├── wvp.js              (generated) Emscripten JS glue
│   └── wvp.wasm            (generated) WASM binary
├── src/
│   ├── decoder.h           C API surface
│   ├── decoder.cpp         FFmpeg demux + decode + sws YUV conversion
│   ├── FileAVIO.h    AVIOContext backed by JS File (streaming reads)
│   └── FileAVIO.cpp  EM_ASYNC_JS file bridge + JS registry
├── CMakeLists.txt          Emscripten build config (Asyncify, LTO, exports)
├── build_ffmpeg.sh         FFmpeg → WASM cross-compile script
└── .clangd                 clangd config for IDE support

Supported Formats

Container	Video	Audio
MP4 / MOV	H.264, HEVC, MPEG-4	AAC, MP3
MKV / WebM	VP8, VP9, AV1	Opus, Vorbis
TS	H.264, MPEG-4	AAC, MP3
DASH (MPD)	any of the above	any of the above

Known Limitations

• 2 GB file size limit (AVIO read offset uses 32-bit int)
• HDR / 10-bit tone-mapped to 8-bit by libswscale (colours may appear slightly washed out)
• DASH: only the lowest-bandwidth video representation is selected