C++ vs Rust vs Go for Media Codec and Media Pipeline Engineering

1. Purpose

This document compares C++, Rust, and Go across two areas:

1. Core codec math — the actual encode/decode inner loops (transforms, motion estimation, entropy coding).
2. Everything around the codec — frame delivery, capture, streaming, post-processing, container parsing, and dynamic library loading.

The goal is to identify which language fits which layer of a real media pipeline, rather than declaring one language a universal winner.

---

2. Memory Model and Stack Allocation

2.1 C++

Manual control over stack vs. heap. Local variables, fixed-size arrays, and structs are stack-allocated by default. This is ideal for scratch buffers used in transform/quantization blocks (e.g., an 8x8 or 16x16 macroblock buffer) because allocation cost is effectively zero and lifetime is fully deterministic.

2.2 Go

Go appears to give the same stack-allocation option (local variables, arrays), but the compiler's escape analysis decides at compile time whether a value can safely stay on the stack or must be promoted to the heap and managed by the garbage collector. Any value whose lifetime the compiler cannot prove is fully local (e.g., it's passed to an interface, stored in a closure, or its address escapes the function) is heap-allocated. This makes stack usage in Go advisory, not guaranteed — a poor fit for latency-sensitive per-frame/per-block buffers where predictable allocation behavior matters.

2.3 Rust

Rust behaves like C++ by default. Local variables, structs, and fixed-size arrays ([T; N], including const-generic sizes) are stack-allocated unless a type explicitly wraps heap storage (Box<T>, Vec<T>, Rc<T>, etc.). There is no hidden escape-analysis pass reassigning memory location behind the programmer's back — the type used is the memory layout used.

Rust also uses RAII (via the Drop trait), giving deterministic destruction identical in spirit to C++ destructors, and — critically — no garbage collector, so there is no GC-pause jitter to worry about in real-time paths.

Where Rust is harder than C++: graph-like decoder state (reference frame buffers pointing at other buffers, motion vectors referencing prior frames) doesn't map cleanly onto Rust's ownership/borrowing rules. C++ solves this with raw or shared pointers; Rust typically requires redesigning the structure around index-based arenas, Rc<RefCell<>>, or similar patterns. This is a real learning curve and occasionally costs some elegance versus "just point at it."

Where "safe Rust" costs a little in the hot loop: bounds-checked indexing can add overhead unless LLVM elides the checks or the code explicitly drops into unsafe (raw pointers, get_unchecked, SIMD intrinsics). In practice, codec authors write unsafe deliberately in the innermost 5% of the code — an opt-in, localized version of what C++ does everywhere by default.

2.4 Summary Table

Property	C++	Rust	Go
Stack allocation	Manual, guaranteed	Guaranteed by type system	Advisory — escape analysis decides
GC pauses	None	None	Present
Deterministic destruction	Yes (destructors)	Yes (Drop / RAII)	No (GC-driven)
Graph-like structures (ref frames)	Easy (raw/shared pointers)	Harder (arenas, Rc<RefCell<>>)	Easy but GC-managed
Hot-loop control	Full, always unchecked	Full, via opt-in unsafe	Limited

---

3. Other Cross-Cutting Considerations for Codec Work

Concern	C++	Rust	Go
SIMD / intrinsics	Mature, decades of hand-tuned code	Stabilizing (std::arch), same LLVM backend, closing the gap	Weak — little autovectorization
Memory safety	Manual, historically a major CVE source in codecs	Compile-time guaranteed, no runtime cost	Safe, but cost is GC
FFI to C libs (ffmpeg, libaom, libvpx)	Native	Near-zero-cost, easy ABI match	cgo has real per-call overhead
Custom allocators / arenas	Mature (placement new, arena allocators)	Good (allocator API, bumpalo)	Essentially none exposed
Concurrency	Manual thread pools, precise data locality	Same, plus compile-time data-race prevention (Send/Sync)	Goroutines excellent for orchestration, weak for tight numeric loops
Codec ecosystem	Dominant (ffmpeg, x264/x265, libaom, libvpx)	Growing (rav1e, symphonia)	Essentially unused for inner loops
WASM target	Mature via Emscripten	Mature, including SIMD	Works, heavier runtime

Real-world examples: rav1e (Rust AV1 encoder), symphonia (pure-Rust audio decoders), and a general trend toward Rust for media parsers specifically because of memory safety against adversarial input.

Bottom line: the reason codecs remain mostly C/C++ (with Rust encroaching) rather than Go isn't just stack vs. heap — it's the combination of no-GC determinism, SIMD access, and low-overhead FFI into the existing C codec ecosystem. Rust matches C++ on the first two and is close enough on the third to be a realistic alternative, especially on the parsing side where memory-safety bugs have historically been the costliest.

---

4. WASM SIMD: Why Rust Has an Edge

When targeting WebAssembly specifically, Rust's advantage is about directness and safety of access, not raw throughput — the WASM SIMD ceiling is a platform limit both languages hit equally.

1. Direct, stable intrinsics. Rust's core::arch::wasm32 exposes the v128 type and instructions (i32x4_add, f32x4_mul, shuffles, etc.) as stable stdlib functions compiling straight to WASM SIMD opcodes via LLVM's wasm32 backend — no emulation shim, and the same mental model as core::arch::x86_64.

2. No runtime feature detection needed. WASM SIMD is a single fixed 128-bit baseline ISA across all supporting engines — unlike native x86, where C++ codecs typically ship multiple dispatch paths (SSE2/AVX2/AVX-512) selected via cpuid at runtime. Both languages benefit here, but Rust's #[cfg(target_feature)] plus intrinsics for a single build target is slightly more ergonomic to keep clean of leaked non-wasm code paths.

3. Portable SIMD without codegen penalty. Crates like wide, or the (nightly) std::simd portable-SIMD API, allow width-generic code that monomorphizes down to simd128 instructions with no abstraction overhead — useful when the same source also targets native SIMD.

4. Lighter toolchain. cargo build --target wasm32-unknown-unknown produces a small binary directly. C++ typically goes through Emscripten, bringing its own libc/runtime shims and larger output unless carefully trimmed.

5. Safety in the surrounding code. The SIMD intrinsics themselves are unsafe in both languages — there's no way around raw lane manipulation. But the scalar glue code around the hot loop (bitstream parsing, buffer indexing, frame-reference bookkeeping) stays memory-safe in Rust by default. Since historical codec CVEs cluster in that surrounding logic rather than the vectorized math itself, this matters directly for a WASM module processing untrusted network input.

Caveat: well-written C++ using WASM SIMD intrinsics hits the same 128-bit throughput ceiling as well-written Rust. There is no inherent Rust speed advantage once both are compiling to simd128 — the win is ergonomics and safety of the surrounding code, not the fast path itself.

Go on WASM: essentially no usable SIMD story and higher runtime overhead — not a serious contender for this workload.

---

5. Beyond the Codec: The Rest of the Media Pipeline

Most of a real media pipeline is I/O, orchestration, and adversarial-input parsing rather than raw pixel throughput. The language choice should be made per layer, not once for the whole system.

5.1 Delivering Decoded Frames to the Screen Surface

Requires zero-copy paths into the GPU (DMA-BUF, IOSurface, D3D11 textures, Vulkan/Metal swapchains) and consistent vsync-deadline delivery.

• C++: most direct, native access to every platform's swapchain/surface API.
• Rust: reaches the same zero-copy paths via bindings (wgpu, ash, windows-rs, objc2) — capable, one FFI layer removed from the native SDK.
• Go: poor fit. Presentation is a hard real-time deadline (16.6ms at 60fps); a GC pause landing at the wrong moment drops a frame. This is one of the clearest cases where "no GC" is a hard requirement, not a nice-to-have.

5.2 Camera Input to Encoder

Mostly buffer-pool management: acquire a buffer from the driver (V4L2 / AVFoundation / Camera2), hand it to the encoder, return it promptly.

• C++: standard, direct.
• Rust: RAII maps naturally onto "return buffer to driver pool on drop," avoiding a classic C++ bug (forgetting to release a camera buffer and stalling capture). A safety benefit, not just a performance one.
• Go: viable for orchestration, but repeated large-buffer copies across the cgo boundary carry real overhead, and GC jitter can show up as capture timestamp inconsistency.

5.3 Encoding Screen Capture

Same shape as camera capture, via DXGI Desktop Duplication, ScreenCaptureKit, or the Linux PipeWire portal — ideally staying zero-copy on the GPU.

• C++: deepest, most mature bindings to all three platforms.
• Rust: closing the gap quickly — windows-rs (Microsoft-maintained) covers Windows, objc2 covers ScreenCaptureKit, ashpd covers the Linux portal.
• Go: essentially unused at this layer, for the same GC-jitter reasons as above.

5.4 Video Streaming (Video Calling, Live Broadcasting)

This layer must be split into two distinct jobs with very different requirements:

• Signaling / control plane (room management, negotiation, presence, REST/gRPC APIs): Go is a genuinely strong choice here, not a compromise. Goroutines and the stdlib networking stack are excellent for this. This is why Pion (a full WebRTC stack in Go) and LiveKit's SFU are written in Go.
• Media plane / jitter buffer / per-packet RTP handling: latency-sensitive, and this is where GC pauses translate directly into audible or visible jitter. Discord's well-documented move from Go to Rust for their voice backend was driven by exactly this — Go worked fine everywhere except the live packet path. C++ (via libwebrtc) and Rust (via str0m, webrtc-rs) are the right choices when the packet path needs deterministic timing.

Takeaway: don't pick one language for "video calling" — pick per-layer.

5.5 Post-Media Processing (Open → Reform → Re-encode)

Batch/offline transcoding is throughput-bound, not latency-bound — nothing is watching a clock, so GC pauses genuinely don't matter.

• Go: legitimately fine. Most of this work is shelling out to ffmpeg, managing a job queue, and parallelizing across files/workers.
• C++ / Rust: only pull ahead when writing custom filters inline rather than orchestrating existing codec binaries. gstreamer-rs and ffmpeg-next cover this well in Rust, with memory safety as a bonus when input files are untrusted.

5.6 Parser (MP4 Muxer / Demuxer)

The strongest case for Rust on this entire list. Container parsing means walking nested, variable-length, attacker-controlled binary structures (boxes/atoms whose size fields can lie) — historically one of the largest CVE sources in libavformat and browser demuxers (integer overflows, out-of-bounds reads from malformed size fields).

• Rust: memory safety plus enum-based modeling of box/atom types fits this problem shape unusually well. This isn't hypothetical — Firefox replaced its C++ MP4 parser with mp4parse-rust specifically because of this CVE history.
• Go: could technically be memory-safe here too, but ecosystem and performance culture for this niche has concentrated in Rust and C++, not Go.
• C++: still dominates by ecosystem size (ffmpeg's demuxers), but new "parse untrusted media" projects are strong Rust candidates regardless of what encodes/decodes the actual samples.

5.7 Latency Loading the Codec Dynamic Library (.so)

Load latency is driven by relocations, symbol table size, and static initializers run at load time — not primarily by which language calls dlopen.

• C++: shared libraries are often slower to load than expected due to global constructors (codecs statically self-registering at load, a classic ffmpeg pattern), RTTI tables, and heavier mangled-symbol tables.
• Rust: cdylib exports have no implicit global-constructor mechanism by default, so a Rust .so tends to be lighter at load unless the crate deliberately opts into something like the ctor crate.
• Go: the worst case here. Loading a Go shared library means bootstrapping the Go runtime itself (scheduler, GC) inside the host process, and Go .so binaries tend to be notably large. This is a real strike against Go independent of the caller's language.
• From the calling side, Rust's libloading and C++'s raw dlopen/dlsym are both negligible-overhead wrappers — the cost lives in the library being loaded, not in the caller.

---

6. Consolidated Recommendation Table

Layer	Best fit	Why
Frame → screen delivery	C++ / Rust	Hard real-time deadline; GC is disqualifying
Camera → encoder	C++ / Rust	Buffer-pool safety, some jitter sensitivity
Screen capture	C++ / Rust	Zero-copy GPU paths
Streaming — control plane	Go (or any)	Not latency-critical; Go's concurrency model shines
Streaming — media/jitter path	C++ / Rust	GC pauses cause audible/visible jitter (Discord's case)
Batch transcode / post-processing	Go / C++ / Rust	Throughput-bound, not latency-bound
Muxer / demuxer	Rust (increasingly)	Adversarial input parsing; safety pays off directly
.so load latency	Rust ≥ C++ > Go	Static-init overhead and runtime bootstrap dominate
Codec inner loops	C++ (Rust close second)	SIMD maturity, FFI, no-GC determinism
Codec inner loops on WASM	C++ ≈ Rust	Same 128-bit ceiling; Rust wins on surrounding-code safety/ergonomics

7. General Pattern

• Anything touching a hard deadline (frame presentation, live audio/video packets) rules out Go, regardless of how convenient it is elsewhere.
• Anything parsing untrusted binary structure (containers, bitstreams) favors Rust, regardless of raw performance differences.
• Anything that is pure orchestration or throughput work (batch jobs, signaling servers, job queues) is a place where Go's simplicity is a genuine win — not a compromise.