FFmpeg is a 20-year-old C codebase that has handled video processing for most of the internet’s history. Getting it to run in a browser, with no server involved, took some real engineering. Here’s how it actually works.
What is WebAssembly?
WebAssembly (WASM) is a binary instruction format that runs in modern browsers at near-native speed. It is not JavaScript — it is a separate, lower-level format compiled from languages like C, C++, and Rust. It runs in the same security sandbox as JavaScript, with no direct hardware access, and typically executes within 10–30% of native speed.
Emscripten is the compiler toolchain that translates C/C++ code into WebAssembly.
How FFmpeg gets compiled to WASM
At a high level:
- FFmpeg’s C source code is fed through Emscripten
- Emscripten translates C into WASM bytecode
- The output is a
.wasmbinary (~30MB for FFmpeg with common codecs) - A JavaScript glue layer provides the FFmpeg API and manages the WASM module lifecycle
The @ffmpeg/ffmpeg npm package is the result of this process, maintained by the community and updated to track FFmpeg releases.
The virtual filesystem
FFmpeg was designed to read and write files. In a browser, there is no direct filesystem access. The solution is a virtual filesystem (VFS) implemented in WASM memory.
When you compress a video on smaller.video:
- Your video is written into the WASM VFS:
ffmpeg.writeFile('input.mp4', videoData) - FFmpeg runs and reads from / writes to this VFS
- The output file is read back out:
ffmpeg.readFile('output.mp4')
The VFS exists only in the browser tab’s memory and is never persisted to disk.
Why SharedArrayBuffer is required
FFmpeg uses multi-threading for performance. In WebAssembly, multi-threading requires SharedArrayBuffer — a shared memory object that can be accessed simultaneously by the main thread and Web Workers.
SharedArrayBuffer was disabled in 2018 due to the Spectre side-channel attack, then re-enabled in 2020 with strict conditions: the page must be “cross-origin isolated.”
Cross-origin isolation: COOP and COEP
Cross-origin isolation requires two HTTP response headers:
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corpCOOP prevents other origins from getting a reference to the window, isolating the browsing context. COEP requires that all resources loaded by the page opt in to cross-origin embedding via Cross-Origin-Resource-Policy headers or CORS. Together they create an environment where SharedArrayBuffer can safely be re-enabled.
For smaller.video, these headers are set on the server. If you are self-hosting a similar tool with Nginx:
add_header Cross-Origin-Opener-Policy "same-origin";
add_header Cross-Origin-Embedder-Policy "require-corp";The ffmpeg.wasm architecture
Browser tab
│
├── Main thread (JavaScript)
│ ├── ffmpeg.js (API wrapper)
│ └── Your application code
│
└── Web Worker (WASM execution)
├── ffmpeg-core.wasm (~30MB binary)
├── Virtual filesystem (your video in memory)
└── ffmpeg-core.js (Emscripten glue)
│
└── SharedArrayBuffer ←→ Main thread communicationThe FFmpeg WASM module runs in a Web Worker to avoid blocking the main thread. The SharedArrayBuffer enables efficient communication between the worker and the main thread, including progress reporting.
Performance characteristics
WASM FFmpeg is fast, but not as fast as native:
| Task | Native FFmpeg | WASM FFmpeg |
|---|---|---|
| 1080p H.264 encode, CRF 23 | ~60fps | ~20–30fps |
| 720p VP9 encode | ~30fps | ~8–15fps |
| Audio-only re-encode | Near-instant | Near-instant |
Encoding 1 minute of 1080p video takes roughly 1–3 minutes in the browser on a modern machine. For most compression use cases, that’s acceptable.
Why it matters for privacy
Traditional browser-based video tools upload your file to a server for processing. That means your video data travels over the network, gets processed on hardware you don’t control, and may be logged or retained.
With browser-side WASM, the encoding happens entirely on your device. The tool provider has no access to your files and no server infrastructure to breach. For confidential recordings — corporate meetings, personal content — that distinction matters.