2025/01/23
Node.js on Alpine vs Debian: Performance differences
- What is Alpine?
- What is Debian?
- Who cares about the standard C library used in a Docker container?
- A bit of history on musl vs glibc
- Breaking down the performance differences
- Conclusions
A few days ago, I was working on a project that required me to use Node.js. I was helping a colleague set up the Docker container used to run the project, and we had to choose between using the Alpine or Debian image for my Docker container. I was heavily against using Alpine, and had to explain why to my colleague. I decided to write this article to share my stance and explain the rationale behind it, and as an exercise in collecting detailed proof of my POV on the matter.
Versions used throughout:
node:24-trixie(Debian 13, glibc 2.41) andnode:24-alpine(Alpine 3.22, musl 1.2.5), x86-64. All numbers come from those sources, not from vibes.
What is Alpine?
Alpine Linux is a lightweight Linux distribution that is designed with security
and simplicity in mind. It is known for its small size and minimalistic design,
making it a popular choice for containerized applications. Alpine Linux uses
musl as its standard C library, which is known for its small size and
simplicity.
What is Debian?
If you don’t know what Debian is, consider a career in carpentry. However, for the sake of
completeness: Debian is a popular Linux distribution that is known for its
stability and large package repository. It is widely used in production
environments and is a popular choice for server deployments. C code compiled on
Debian systems is compiled against glibc, the GNU C library, which is known
for its feature-richness and compatibility with a wide range of software.
Who cares about the standard C library used in a Docker container?
You might’ve read the previous sections and thought, “Who cares about the standard C library used in a Docker container? I’m not a kernel developer, I’m just trying to run my Node.js application.” And you’d be right. For most applications, the choice of standard C library doesn’t matter. However, if you know me at all, you know that I care about the nitty-gritty low-level details of software development. And I won’t let my application run on anything that I ignore the implementation details of.
Node.js is a C++ application at its core, and it interacts with the operating system through system calls and the standard C library. The choice of standard C library can have an impact on the performance of the application, especially when it comes to I/O-bound operations, and thinking that a JIT-compiled language like JavaScript is immune to these low-level details is naive.
Syscalls and I/O operations are the most common ways that a Node.js application
interacts with the operating system. glibc’s implementations of most of them
(I’m talking malloc, memcpy, printf, etc.) are heavily optimized: memcpy
alone has hand-written AVX2 and AVX-512 variants picked at runtime for your CPU.
malloc, for example, is implemented in a way that makes it fast and efficient
for allocating and deallocating memory, and differs from musl in several
aspects.
Why is that important? Well, malloc is our key to the magical heap.
Heap? What’s that?
The heap is essentially a list of memory regions that a running program uses to store data on, dynamically. When we access memory on the heap, the speed is OFC slower than when handling the stack, because in a stack frame data is simply pushed/popped sequentially: there’s no need to go and look for a particular memory address. On the heap, however, the allocator must find a free memory block that is large enough to store the data. I don’t need to tell you that this is a much slower process than the stack operation, so the way memory allocators optimise the de/allocation of memory blocks is crucial for the performance of the program.
Since the stack is much faster than the heap, then why we would even need the heap? Because sometimes we need to allocate memory dynamically in the program, with a size or a lifetime the compiler can’t know in advance. The heap is designed to cope with exactly that.
Wait, does JavaScript even call malloc?
Here’s the twist most “Alpine is slow” posts miss: most of your JavaScript
never touches malloc at all. Objects, strings, arrays, closures: all of
them live on V8’s own garbage-collected heap. V8 asks the kernel for big chunks
of pages with mmap() and manages them itself, with a bump allocator for new
objects and a GC to clean up. libc doesn’t get a say.
What does go through malloc is everything outside that heap: Buffer and
ArrayBuffer contents, zlib’s compression state, OpenSSL’s contexts, libuv
requests, native addons, and V8’s own compiler scratch space. Click around:
const user = { id: 1, name: "Ada" };- Runs on
- main thread
- musl vs glibc matters?
- no
Bump-allocated in V8's young generation. V8 gets its pages straight from the kernel with mmap() and manages them with its own garbage collector. libc never sees this allocation.
const s = "🍕".repeat(1_000_000);- Runs on
- main thread
- musl vs glibc matters?
- no
Large objects go to V8's large-object space, still V8-managed pages. Same for arrays, Maps, closures and the result of JSON.parse().
const b = Buffer.allocUnsafe(100);- Runs on
- main thread
- musl vs glibc matters?
- a bit
Sliced out of a shared pool (Buffer.poolSize: 64 KiB in Node 24, 8 KiB in Node 22) for anything under half the pool size. The pool itself is one malloc'd ArrayBuffer, so libc is only hit when a pool fills up.
const b = Buffer.alloc(256 * 1024);- Runs on
- main thread
- musl vs glibc matters?
- yes
ArrayBuffer and Buffer contents live outside the V8 heap: V8's default allocator calls calloc(). 256 KiB is over both mmap thresholds, but only glibc adapts: after the first free() it serves this size from the heap. musl does mmap() + munmap() every single time.
zlib.gzip(payload, (err, out) => {});- Runs on
- libuv threadpool
- musl vs glibc matters?
- yes
Every deflate stream needs about 256 KiB of state with the default settings (zlib documents it as (1 << (windowBits+2)) + (1 << (memLevel+9))), malloc'd in ~64 KiB pieces. The compression runs on the threadpool, so several streams means several threads in malloc at once.
crypto.pbkdf2(pw, salt, 1e5, 64, "sha512", cb);- Runs on
- libuv threadpool
- musl vs glibc matters?
- a bit
OpenSSL allocates its contexts with OPENSSL_malloc(), i.e. malloc(), on a threadpool thread. Allocation is small next to the hashing itself; this one is mostly CPU-bound.
const data = await fs.promises.readFile("big.json");- Runs on
- libuv threadpool
- musl vs glibc matters?
- yes
The read() calls happen on the threadpool and the bytes land in a Buffer backing store from calloc(). A multi-megabyte file is a fresh mmap() on musl. Parsing it afterwards with JSON.parse() is pure V8 heap.
await sharp(input).resize(800).toBuffer();- Runs on
- addon's own threads
- musl vs glibc matters?
- yes
libvips (behind sharp) splits images into tiles across its own thread pool and allocates constantly. This is the worst case for a single global lock. Also, you need a musl build of the addon: sharp ships separate linuxmusl binaries.
for (let i = 0; i < 1e6; i++) total += price(items[i]);- Runs on
- V8 background threads
- musl vs glibc matters?
- a bit
Your objects stay on the V8 heap, but once price() is hot TurboFan compiles it on a background thread, and the compiler's scratch memory (zones) comes from malloc(). The same goes for parts of the GC's bookkeeping.
So the question isn’t “is musl slower?”, it’s “how much of my workload is the
second list?”. For a typical API server shovelling JSON around, less than you’d
think. For anything doing compression, crypto, file I/O, image processing or big
Buffers, a lot.
A bit of history on musl vs glibc
musl and glibc are two different implementations of the standard C library,
and they have different design philosophies.
glibc has been around for a long time (since 1987, probably older than me and
you) and is known for its feature-richness and compatibility with a wide range
of software. It’s accumulated a ton of triage and consequent optimizations over
the years, and it’s the default C library on most Linux distributions. However,
glibc is also known for its large size and memory footprint, which can be a
problem for resource-constrained environments.
musl, on the other hand, is a newer (its first release was in 2011)
implementation of the standard C library
that is designed with simplicity and lightweightness in mind. It is known for its
small size and minimalistic design, and it is optimized for static linking,
which can result in developers being able to ship applications as small, single
binaries with faster startup times, keeping their runtime memory footprint low.
A godsend for embedded systems and IoT devices. musl is the
default C library on Alpine Linux, and has become a popular choice for containerised
applications because of its minimalistic design philosophy.
However, musl is also known for its lack of compatibility with
some software that expects glibc-specific behavior (or maybe glibc is known
for doing sh!t its own way) and for putting simplicity
over tunability, and this has a certain impact on the performance of some
applications compiled against it.
Node itself takes musl seriously these days: linux-x64-musl is an official
Tier 2 platform, built and published on nodejs.org, and that’s exactly the
tarball the node:alpine images download. So we’re comparing two supported
builds, not a hack against the real thing.
Breaking down the performance differences
Well, now you know the difference between musl and glibc, and you might’ve
guessed that the choice of standard C library is what makes the difference in
performance between Alpine and Debian. But how does it actually affect the
performance of a Node.js application on a practical level? The internet is full
of theories with no practical examples, but you know this blog’s existence is
entirely devoted to clearing the fog of confusion around software development, so
let’s provide some straightforward explanations.
glibc
glibc uses ptmalloc2 (“pthreads malloc”), a derivation of Doug Lea’s
dlmalloc that is optimized for multi-threaded applications. On top of it,
since glibc 2.26, sits the tcache: a small per-thread cache.
- tcache: 64 bins for requests up to 1,032 bytes, 16 bytes apart, up to 7
chunks each. A
mallocthat hits the tcache is a handful of instructions and takes no lock at all, because the cache belongs to the calling thread. Afreeputs the chunk back in the same place. - fastbins: small chunks (up to 128 bytes) kept in LIFO lists, so a recently freed chunk of the right size is reused immediately.
- small and large bins: below 1 KiB, exact-size lists; above, sorted lists searched for the best fit.
- mmap: anything above the mmap threshold gets its own mapping straight from the kernel. The threshold starts at 128 KiB but is dynamic: when you free an mmap’d chunk, glibc raises the threshold to that chunk’s size (up to 32 MiB), on the theory that you’ll ask for that size again and it’s cheaper to keep it in the heap.
The big difference between ptmalloc and dlmalloc, though, is in support for
concurrent allocations. The ptmalloc allocator maintains multiple
arenas.
Each arena is an independent instance of the allocator protected by its own lock.
Upon allocation, the thread invoking the allocator attempts to lock an
arena, starting with the one it used last. If an arena is locked successfully,
it is used to satisfy the allocation request, otherwise another one is tried,
or a new arena is created, up to 8 per CPU core on 64-bit systems. Between the
tcache and the arenas, ptmalloc’s architecture avoids lock contention as much
as possible.
musl
musl has used mallocng since version 1.2.1 (2020, so any Alpine from 3.13
onwards). It was written from scratch with very different priorities from
ptmalloc: hardening first, then low fragmentation and returning memory to the
OS, with raw speed further down the list.
- Allocations are rounded up to one of 48 size classes (16 bytes up to ~128 KiB) and handed out as slots from groups of same-size slots.
- The allocator’s metadata lives out-of-band, in separate areas away from your data, so a heap buffer overflow can’t easily corrupt the allocator’s own bookkeeping. That’s a real security win.
- There is no per-thread cache, and there’s a single global lock,
__malloc_lock, taken on everymallocandfreeas soon as your process has a second thread. Spoiler: a Node.js process always does. - Anything of 131,052 bytes or more gets its own
mmap(), andfree()munmap()s it immediately. There’s no adaptive threshold: allocate and free a 256 KiB buffer in a loop and you pay two syscalls and a round of page faults every iteration.
Here’s the same request going through both. Try the presets, and in particular compare 256 KiB with and without glibc’s raised threshold:
- tcacheper-thread cache, ≤ 1032 B, 7 chunks per size
- fastbinchunks ≤ 128 B, LIFO
- smallbinchunks < 1 KiB, exact size
- largebinchunks ≥ 1 KiB, best fit
- top chunkgrow the arena: brk() or a new heap
- mmap()its own mapping, unmapped on free()
- Reserved
- Overhead
- Lock
- Syscalls
- global lock__malloc_lock, taken on every call once a 2nd thread exists
- size class48 classes, 16 B to 128 KiB
- group slota free slot in a group of same-size slots
- new groupcarved from a bigger slot, or mmap()
- mmap()≥ 131,052 B: own mapping, unmapped on free()
- Reserved
- Overhead
- Lock
- Syscalls
The efficiency chart is the part I find the most interesting. musl’s coarser size classes waste a bit more memory in places (a 64 KiB request lands in a ~73 KiB slot), glibc’s 16-byte granularity wastes less, and both converge as sizes grow. Neither is dramatically better here. Where they really differ is locking.
The lock
A single global lock is perfectly fine if only one thread allocates. It’s a
problem when many threads allocate at the same time, because they queue up behind
each other no matter how many cores you have. Below, every thread does some work,
calls malloc, and repeats. I gave both allocators the exact same cost per call,
so the only thing being modelled is contention:
With a few threads and enough work between allocations, the two lines are
indistinguishable. Push the thread count up or the work down and musl flattens
out: it can never complete more than one malloc per critical section, however
many cores you throw at it. glibc keeps scaling, because each thread has its own
tcache and, on a miss, its own arena.
And in Node, those threads are real. The libuv threadpool (4 threads by default,
UV_THREADPOOL_SIZE to change it) runs your fs, zlib, crypto and
dns.lookup work, V8 runs GC and JIT compilation on background threads, and
native addons like sharp bring their own thread pools. That’s where the
global lock shows up in profiles.
About those thread stacks
You’ll often read that musl’s small default thread stack makes it unsuitable
for Node. That one is half true, and I used to repeat it too, so let’s be
precise. musl’s default for a new thread is 128 KiB (DEFAULT_STACK_SIZE is
literally 131072), where glibc’s default comes from RLIMIT_STACK, usually
8 MiB. But Node rarely relies on the default: libuv asks for RLIMIT_STACK
explicitly, worker_threads set their own 4 MiB, and V8 caps JS recursion at
about 984 KB regardless. The tiny default only hits threads that native code
creates with a bare pthread_create():
When it does hit, it isn’t a slowdown: it’s a segfault in an addon that worked perfectly on Debian. That’s arguably worse.
Conclusions
With all that said, it’s clear that the choice of standard C library can have a significant impact on the performance of a Node.js application, though not necessarily in the place people expect.
Most web applications do heavy string processing, I/O operations, and/or depend on delegating workloads to native dependencies. The string processing mostly stays on V8’s heap and doesn’t care about your libc. The I/O, compression, crypto and native work don’t: they allocate outside the V8 heap, often from several threads at once, and that’s where musl’s single lock and fixed mmap threshold cost you.
In general, if you’re running a Node.js application and want to maximize its
runtime performance, with a lot of threadpool or native work, then glibc is
the way to go. If you’re running a Node.js application in a resource-constrained
environment and want to minimize its image size and memory footprint, then
musl is probably the way to go for you. And if you’re stuck on Alpine but hit
the lock, you can swap the allocator without changing distro: LD_PRELOAD a
multi-threaded allocator like jemalloc or mimalloc. Either way, measure your own
workload before trusting anyone’s blog post. Including this one.