WebAssembly, WebCodecs & WebGPU

1. W3C WebAssembly Standards (TR)

The W3C TR WebAssembly filter lists six documents covering three specs (each in v1 and v2):

Spec	Status
WebAssembly Core Specification	Standard
WebAssembly JavaScript Interface	Standard
WebAssembly Web API	Standard

These cover the binary format, JS bindings, and streaming compilation only — not media or graphics APIs.

---

2. WebGPU Status

Item	Detail
Spec	w3.org/TR/webgpu
Status	Candidate Recommendation Draft (CRD), last updated May 2026
Shading language (WGSL)	Also CRD as of May 2026
Browser support	All major browsers (Chrome, Safari, Firefox); >84% globally
Governing body	W3C GPU for the Web Working Group

Note: WebGL is governed by Khronos Group, not W3C. WebGPU is the modern W3C-standardized successor for GPU access on the web.

---

3. Video Frame Rendering with WebGPU

WebGPU is well-suited for video frame rendering, especially when combined with WebCodecs:

<video> or VideoDecoder (WebCodecs)
       ↓
  VideoFrame
       ↓
importExternalTexture()   ← zero-copy GPU import
       ↓
  Render Pipeline (WGSL shaders)
       ↓
  GPUCanvasContext (<canvas>)

Key advantage: importExternalTexture()

• Avoids CPU-side pixel copying by importing decoded frames directly to GPU memory
• Enables full shader-based post-processing in the same pass:
  • Color space conversion, tone mapping (HDR)
  • Chroma keying / green screen
  • Real-time filters, overlays, multi-video compositing

WebGPU vs WebGL for video

Feature	WebGL	WebGPU
Video texture import	texImage2D(video)	importExternalTexture() ✓
Zero-copy path	Limited	Yes
Compute shaders on frames	No	Yes
WebAssembly integration	Indirect	Designed for it

---

4. WebAssembly Direct Access to WebGPU

Short answer: Not yet — JavaScript is still required as the bridge.

Current reality

C++ / Rust code
      ↓  (compiled by Emscripten / wasm-bindgen)
  WebAssembly module
      ↓  (JS glue code, auto-generated)
  JavaScript WebGPU API
      ↓
     GPU

The Wasm module calls WebGPU through auto-generated JS bindings — not natively.

Active proposals moving toward direct access

Proposal	Purpose
WebIDL Bindings (→ Component Model)	Wasm calling Web APIs without JS glue overhead
Component Model	Structured interface types between Wasm and host
Memory64	Removes 4GB linear memory limit for large GPU workloads
WASI	Future extensions may include GPU access

---

5. WebIDL Bindings Proposal: Where It Went

The proposal evolved through a chain of repos before being absorbed:

Stage	Repo	Status
WebIDL Bindings (original)	github.com/WebAssembly/webidl-bindings	Archived
Interface Types	github.com/WebAssembly/interface-types	Archived
Component Model (current)	github.com/WebAssembly/component-model	Active, Phase 1
Direct Web API access from Wasm	No formal proposal yet	Under discussion

Follow today: github.com/WebAssembly/component-model and component-model.bytecodealliance.org

Component Model phase outlook

• Expected to advance to Phase 2 after WASI 0.3 release (released Feb 2026)
• Phase 5 (standard) likely after WASI 1.0, expected late 2026 / early 2027

---

6. WebCodecs + WebAssembly Interop

WebCodecs spec status

Item	Detail
Spec	w3.org/TR/webcodecs
Governing body	W3C Media Working Group
Browser support	Chrome, Edge (full); Firefox, Safari (improving throughout 2025–2026)

Current interop path (always via JS)

VideoDecoder (WebCodecs, JS)
       ↓  output callback
  VideoFrame (JS object)
       ↓  videoFrame.copyTo(arrayBuffer)   ← copy #1
  ArrayBuffer (JS memory)
       ↓  write into Wasm linear memory
  WebAssembly processes pixels
       ↓  write result back to ArrayBuffer
  new VideoFrame(buffer)                   ← copy #2
       ↓
  VideoEncoder / Canvas / WebGPU

The core problem: mandatory double copy

• VideoFrame.copyTo() always copies — no mechanism to transfer ownership
• new VideoFrame(buffer) creates a second copy
• Full HD frame copy cost: ~10–20ms on desktop, ~40–60ms on mobile
• Frame budget: only 40ms at 25fps, 20ms at 50fps — very tight

What's being discussed (no formal proposal yet)

Idea	Status
Memory mapping for Wasm	Discussed in WICG reducing-memory-copies repo; not yet a formal proposal
Zero-copy via WebGPU	Partial: importExternalTexture() avoids CPU copy when going GPU→GPU
SharedArrayBuffer	Available now; enables parallel worker processing but doesn't eliminate copies

---

7. Practical Recommendations (2026)

Goal	Best path today
Video decoding + GPU rendering	WebCodecs → importExternalTexture() → WebGPU
CPU-side frame processing in Wasm	WebCodecs → copyTo() → Wasm linear memory (via SharedArrayBuffer for parallelism)
Avoid shipping codec in Wasm	Use WebCodecs (3× faster than ffmpeg.wasm in benchmarks)
Follow zero-copy Wasm progress	Watch github.com/WebAssembly/component-model and W3C CG meetings

---

8. Key Links

Resource	URL
W3C WebAssembly TR filter	https://www.w3.org/TR/?filter-tr-name=WebAssembly
WebGPU spec	https://www.w3.org/TR/webgpu/
WebCodecs spec	https://www.w3.org/TR/webcodecs/
Component Model repo	https://github.com/WebAssembly/component-model
Component Model docs	https://component-model.bytecodealliance.org/
WICG reducing-memory-copies	https://github.com/WICG/reducing-memory-copies
WebGL (Khronos)	https://registry.khronos.org/webgl/specs/latest/