ffmpegwebassemblytechnical

ffmpeg.wasm explained: how FFmpeg runs in the browser

How ffmpeg.wasm runs FFmpeg inside a browser tab: the WebAssembly build, the in-memory filesystem, and why multi-threading needs COOP and COEP headers.

By Kyllian · Published · Updated · 5 min read

FFmpeg is a 20-year-old C codebase that has handled video processing for most of the internet’s history. ffmpeg.wasm is that same codebase compiled to WebAssembly, so it runs inside a browser tab with no server involved, and it is what the compressor on this site is built on. Getting there took some real engineering. Here’s how it actually works.

What is WebAssembly?

WebAssembly (WASM) is a binary instruction format that modern browsers execute directly. It is not JavaScript — it is a separate, lower-level format compiled from languages like C, C++ and Rust, and it runs in the same security sandbox as JavaScript with no direct hardware access. It is substantially faster than JavaScript for compute-heavy work like video encoding, though how close it gets to native code depends heavily on the workload, and a browser sandbox will never match a bare-metal binary.

Emscripten is the compiler toolchain that translates C/C++ into WebAssembly.

How FFmpeg gets compiled to WASM

At a high level:

  1. FFmpeg’s C source code is fed through Emscripten
  2. Emscripten translates C into WASM bytecode
  3. The output is a .wasm binary (~30MB for FFmpeg with common codecs)
  4. A JavaScript glue layer provides the FFmpeg API and manages the WASM module lifecycle

The @ffmpeg/ffmpeg npm package is the result of this process, maintained by the ffmpeg.wasm project and updated to track FFmpeg releases. It is what powers the compressor on this site.

The virtual filesystem

FFmpeg was designed to read and write files. In a browser, there is no direct filesystem access. The solution is a virtual filesystem (VFS) implemented in WASM memory.

When you compress a video in your browser here:

  1. Your video is written into the WASM VFS: ffmpeg.writeFile('input.mp4', videoData)
  2. FFmpeg runs and reads from / writes to this VFS
  3. The output file is read back out: ffmpeg.readFile('output.mp4')

The VFS exists only in the browser tab’s memory and is never persisted to disk.

Why SharedArrayBuffer is required

FFmpeg uses multi-threading for performance. In WebAssembly, multi-threading requires SharedArrayBuffer — a shared memory object that the main thread and Web Workers can both access.

Browsers disabled SharedArrayBuffer in response to the Spectre class of side-channel attacks, and later restored it under a strict condition: the page must be cross-origin isolated. Without that, the tool has to fall back to a single-threaded build of the FFmpeg core — slower, but still entirely local.

Cross-origin isolation: COOP and COEP

Cross-origin isolation requires two HTTP response headers:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

COOP prevents other origins from getting a reference to the window, isolating the browsing context. COEP requires that every resource the page loads opts in to cross-origin embedding, via a Cross-Origin-Resource-Policy header or CORS. Together they create an environment where SharedArrayBuffer can safely be exposed again.

This is the single most common reason a self-hosted ffmpeg.wasm deployment fails: the headers are missing, crossOriginIsolated is false, and multi-threading silently never engages.

For smaller.video, these headers are set on the server. If you are self-hosting a similar tool with Nginx:

add_header Cross-Origin-Opener-Policy "same-origin";
add_header Cross-Origin-Embedder-Policy "require-corp";

The ffmpeg.wasm architecture

Browser tab
│
├── Main thread (JavaScript)
│   ├── ffmpeg.js (API wrapper)
│   └── Your application code
│
└── Web Worker (WASM execution)
    ├── ffmpeg-core.wasm (~30MB binary)
    ├── Virtual filesystem (your video in memory)
    └── ffmpeg-core.js (Emscripten glue)
        │
        └── SharedArrayBuffer ←→ Main thread communication

The FFmpeg WASM module runs in a Web Worker to avoid blocking the main thread. The SharedArrayBuffer enables efficient communication between the worker and the main thread, including progress reporting.

Using ffmpeg.wasm in your own page

The API is small. You load a core, write a file into the virtual filesystem, run an ordinary FFmpeg argument list, and read the result back out. With version 0.12 of the packages:

import { FFmpeg } from '@ffmpeg/ffmpeg';
import { fetchFile, toBlobURL } from '@ffmpeg/util';

const ffmpeg = new FFmpeg();
ffmpeg.on('progress', ({ progress }) => console.log(Math.round(progress * 100) + '%'));

const base = 'https://unpkg.com/@ffmpeg/core@0.12.10/dist/esm';
await ffmpeg.load({
  coreURL: await toBlobURL(`${base}/ffmpeg-core.js`, 'text/javascript'),
  wasmURL: await toBlobURL(`${base}/ffmpeg-core.wasm`, 'application/wasm'),
});

await ffmpeg.writeFile('input.mp4', await fetchFile(file)); // file from an <input type="file">
await ffmpeg.exec(['-i', 'input.mp4', '-c:v', 'libx264', '-crf', '28', '-preset', 'veryfast', 'output.mp4']);
const data = await ffmpeg.readFile('output.mp4');
const url = URL.createObjectURL(new Blob([data], { type: 'video/mp4' }));

That is the single-threaded core, which works on any page. The multi-threaded one is a separate package, @ffmpeg/core-mt, which also needs a workerURL pointing at its ffmpeg-core.worker.js, and it only loads when the page is cross-origin isolated. This site tries the multi-threaded core first and falls back to the single-threaded one when isolation is not available. It also serves both cores from its own domain rather than a CDN, because under cross-origin isolation a third-party file either has to opt in (require-corp) or is fetched without credentials (credentialless, which this site uses), and self-hosting removes the question either way.

Two practical notes. The core is tens of megabytes, so load it once and keep the instance around rather than creating one per file. And everything in the virtual filesystem lives in the tab’s memory, so delete input and output files with ffmpeg.deleteFile() once you have what you need, or a second large file can push the tab over its memory limit.

Performance characteristics

WASM FFmpeg is usable but it is not native FFmpeg, and it is worth being honest about the gap rather than quoting a number that will be wrong on your machine.

Four things cost you speed relative to a terminal:

  • The sandbox. WebAssembly runs under browser memory and security constraints that a native binary does not.
  • No hardware encoders. A web page cannot reach your GPU’s dedicated video encoder, so everything runs on the CPU.
  • Thread count. If the page is not cross-origin isolated, SharedArrayBuffer is unavailable and the encode is single-threaded.
  • The encoder you asked for. x264 at a fast preset is the cheap case. x265 and libvpx-vp9 are both in the build behind this site, and both cost several times as much CPU for the same clip, which a browser tab feels far more sharply than a desktop does.

The practical shape of it: an encode that takes seconds in a terminal takes minutes in a tab, and encoding effort scales with resolution and duration, and single-threaded fallback roughly doubles it again. Time a 30-second clip before you commit to a two-hour one. If you compress video daily, install FFmpeg; if you need one file smaller without uploading it anywhere, the browser is fine.

Why it matters for privacy

Traditional browser-based video tools upload your file to a server for processing. That means your video data travels over the network, gets processed on hardware you don’t control, and may be logged or retained.

With browser-side WASM, the encoding happens entirely on your device. The tool provider has no access to your files and no server infrastructure to breach. For confidential recordings — corporate meetings, medical or legal material, anything under NDA — that distinction is the difference between a usable tool and one your policy forbids.

You can verify it rather than trust it: open your browser’s Network tab and compress a file. You will see the site’s assets and the WebAssembly core download, and no request carrying your video.

Where to go next

Sources

Try it on your own file. The encode runs in this tab, so nothing is uploaded.

Try the compressor