ONE SERVER · 10,000 CLIENTS · FOUR RUNTIMES
Ten thousand clients hit one box in the same instant — the 1999 , still the truest test of a concurrency model. We fix one workload and hold that same burst four ways: Java’s virtual threads, Node’s event loop, Go’s goroutines, and Python’s GIL. The scenes play themselves — watch, or just scroll. are tappable.
Ten thousand clients, one server, the same instant. Before we can compare Java, Node, Go, and Python, we need the problem they all answer — the C10K burst — and the one fact that makes it tractable: the work is almost entirely waiting. This chapter sets the stage and previews the only two tricks anyone has ever used.
Ten thousand clients hit one server in the same millisecond. That is the , named by Dan Kegel in 1999 after the Simtel host cdrom.com served 10,000 clients at once over a single gigabit link — back when the whole field assumed a server simply could not. This is the burst we follow end to end: one workload, four runtimes, and the same question every time — where do 10,000 half-finished jobs live, and who wakes them back up?
is dealing with many things at once — a property of how a program is structured. is doing many things at once — executing on more than one core. They are independent: a single-core box can juggle 10,000 clients concurrently with zero parallelism, and a four-core box runs at most four things in true parallel no matter how the code is shaped. Almost everything in this tutorial is about concurrency; parallelism only shows up when the work is CPU-bound.
Look at one request: ~100 ms waiting on a database, an API, or a disk, then ~1 ms of actual CPU. For 99% of its life the worker is parked, waiting for bytes to arrive. That idle time is the entire opportunity — while one request waits, the machine could be serving thousands of others. Whether a task is is the single axis that decides which model wins, and the ordinary web request is overwhelmingly I/O-bound.
How many jobs must you hold at once? answers it: L = λ × W — items in the system equal arrival rate times the time each spends inside. Ten thousand clients arriving together, each lasting ~0.1 s, means you need somewhere to park up to 10,000 paused jobs simultaneously. The hard problem was never “how fast” — it is “where do 10,000 half-done jobs wait, and how much does each waiting slot cost?”
Here is the whole tutorial in advance. There are only two cheap ways to hold 10,000 paused jobs. One: make the waiting free — never dedicate a thread to a job that is only waiting; hand the wait to the kernel and loop over whatever is ready (the ). Two: make the workers free — give each job its own tiny that the runtime multiplexes onto a few real ones. Every runtime ahead is one of these, or both.
Why 10,000 was hard. Early servers gave every connection its own process or thread, and the kernels of the day scanned every socket on every pass with select/poll — O(n) work per event. At a few thousand connections the scanning itself became the bottleneck, long before memory ran out.
What broke the wall. Kernel event APIs that scale with the number of ready sockets instead of the total: (FreeBSD, 2000) and epoll (Linux, 2002). nginx (2004) was built around them and its memory stayed flat as connections climbed, while Apache’s prefork model grew linearly — the same split you will see between event loops and thread-per-request below.
The numeronym lineage. Once 10,000 fell, the goal posts moved: C10M (ten million connections on one box) became a real target in the 2010s, chased by kernel-bypass networking and userspace TCP. The lesson stuck — concurrency is a memory-and-scheduling problem, not a raw-speed one.
The request itself. Everything upstream of the burst — DNS, TCP, TLS — is its own story; the press-Enter tutorial follows a single one of these connections from keypress to pixels.
Before the four runtimes diverge, they share one foundation: the operating-system thread. It is the primitive each of them either rations, hides, or replaces — so it pays to see exactly why you cannot simply spawn ten thousand of them.
The obvious answer to the burst is “one per client.” But a thread is a kernel object with its own pre-reserved stack: ~1 MB on the JVM, a full 8 MB for a default Linux pthread. That memory is committed up front, before the thread runs a single line. The primitive is powerful — it lets you write plain, blocking, top-to-bottom code — but it is not free, and the bill scales with the number of threads.
OS threads are preemptively scheduled: the kernel time-slices, interrupting a thread mid-instruction to run another. Each saves registers, swaps page tables, and pollutes the CPU cache — roughly 1–2 µs of pure overhead, and a round trip into kernel mode. A handful of threads, no problem. Ten thousand of them fighting for a few cores, and the scheduler itself becomes the workload.
Put the two costs together. Accept a connection, spawn a thread, let it block on I/O: readable and simple — until 10,000 threads mean 10–80 GB of stacks plus a scheduler thrashing between them. This is the exact wall the named. It is why Apache’s prefork model grew memory linearly with connections while event-driven nginx stayed flat, and why “just add threads” stops working long before the burst is served.
Here is the specific waste. When a thread calls read() and no data is ready, the kernel parks the whole thread — its multi-megabyte stack and its scheduler slot sit idle, holding a job that is 99% waiting anyway. The rail is occupied but doing nothing. Every model in the next four chapters is an attack on this one line: stop paying for a full just to wait.
With the cost concrete, the fork from the last chapter sharpens into two mechanisms. Either stop giving a waiting job a thread at all — hand the wait to and let one thread loop over whatever is ready (the ). Or keep one-thread-per-job, but make the “thread” a few-hundred-byte the runtime parks and resumes for you. Java and Python will do the first; Go does the second; several do both.
Direct vs indirect switch cost. The ~1–2 µs figure is just the direct work — saving and restoring registers and switching address spaces. The larger, harder-to-measure cost is indirect: a fresh thread finds cold caches and a stale TLB, so the first thousands of memory accesses after a switch run slow. This is why user-space switches (goroutines, virtual threads) that skip the kernel and keep caches warm are an order of magnitude cheaper.
Voluntary vs involuntary. A thread that blocks on I/O yields voluntarily — cheap and cooperative. A thread preempted by the scheduler because its time slice expired is involuntary — and it can be interrupted holding a lock, which is how thread-heavy systems get convoy effects and latency spikes.
Stacks can be tuned, not eliminated. You can shrink a thread stack with ulimit -s or the JVM’s -Xss, and 10,000 threads with 256 KB stacks is only ~2.5 GB — survivable. But you are still paying the kernel scheduler and cache costs; tuning the stack down buys headroom, not a new model.
Java is the purest case, because a Java thread genuinely is an OS thread — so its whole history is the fight to serve more clients than it has threads. It arrives, after 27 years, at the answer that removes the problem entirely: make the thread itself almost free.
Since 1996 a java.lang.Thread has been a thin wrapper over an — a “platform thread”: 1:1, ~1 MB stack, kernel-scheduled. This is Java’s great gift and its ceiling. You get gorgeous blocking code — InputStream.read(), Thread.sleep() — but every thread costs a real OS thread, so 10,000 clients slam straight into the previous chapter’s wall.
If you cannot afford 10,000 threads, you share them. An ExecutorService of, say, 200 threads pulls tasks from a queue. It bounds memory — but a caps concurrency at the pool size. If each task blocks 100 ms on I/O, 200 threads top out near 2,000 requests per second and the other 9,800 clients wait in the queue. You have traded the memory wall for a throughput ceiling.
Before 2023 the scalable Java answer was to stop blocking entirely: NIO, Netty, CompletableFuture, Project Reactor — never block a pooled thread, chain callbacks instead. It reaches C10K, but at a price. Stack traces evaporate, debugging gets hard, and every library in the path must be non-blocking: one accidental blocking call and you have your whole codebase and stalled the loop.
Project Loom’s shipped for real in Java 21 (September 2023, JEP 444). A virtual thread is a few-hundred-byte object whose stack lives on the heap and grows on demand — the JVM can hold millions. They run on a small pool of (about one per CPU). Instead of rationing threads, Java made them so cheap you stop counting.
The trick is the . When a virtual thread hits a blocking call, the JVM unmounts it from its carrier and parks it on the heap; the carrier immediately runs another virtual thread. Your code is plain, blocking, one-thread-per-request — the old readable style — but 10,000 of them cost almost nothing. Thread-per-request is back, and it scales. (Watch for pinning: a synchronized block or native frame can nail a virtual thread to its carrier.)
A continuation plus a scheduler. A virtual thread is really two pieces: a Continuation that captures where the code paused, and a scheduler (a ForkJoinPool of carriers) that decides which carrier runs it next. Mounting copies the continuation’s stack onto a carrier; unmounting copies it back to the heap. That copy is the whole cost, and it is tiny.
Pinning, the sharp edge. Inside a synchronized block or a native call the JVM cannot unmount, so the virtual thread pins its carrier for the duration. Enough pinned carriers and throughput collapses to the carrier count — which is why Loom-era guidance is to prefer ReentrantLock over synchronized on hot blocking paths.
Structured concurrency. Loom’s companion feature, StructuredTaskScope, ties a bundle of virtual threads to a lexical scope: if one fails or the scope is cancelled, its siblings are cancelled too. It makes “fan out to 100 services, wait for all” a few readable lines with no leaked threads.
Node is the event loop in its purest, most famous form: one thread that refuses to wait. It is the answer to “what if blocking were simply not allowed?” — and the four-thread secret and the one-core catch are both consequences of that single choice.
Node takes the opposite bet from Java: instead of many cheap threads, it runs your JavaScript on one thread spinning an (via ). There is no thread-per-connection anywhere. Each of the 10,000 clients is a callback registered on the loop; the single thread never blocks — it asks the OS “who is ready?” and runs their callbacks, one after another, forever.
This only works because the waiting is somebody else’s job. Instead of blocking on read(), Node registers interest with and returns to the loop instantly. When bytes arrive, the kernel marks that socket ready and the loop fires its callback. means the wait costs nothing, because no thread is sitting in it — the same C10K trick nginx used, wearing JavaScript.
The loop is powerful and unreadable, so the syntax evolved to hide it: callbacks (2009) → Promises → (Node 7.6, 2017). The trap everyone falls for once: await does not wait. It registers a and yields control back to the loop, which serves other clients until your awaited result is ready. Same event loop underneath — it just finally reads like ordinary top-to-bottom code.
Not everything has a non-blocking syscall. File I/O, DNS (getaddrinfo), and crypto do not, so runs them on a thread pool instead — default size 4 (UV_THREADPOOL_SIZE, max 1024). Your JavaScript loop stays single-threaded; only these offloaded operations touch threads. Node’s famous “single thread” has always had four helpers in the back room.
The weakness is the flip side of the strength: it is all one thread. A single callback — a giant JSON parse, a bcrypt hash, a tight loop — blocks the loop, and all 10,000 clients freeze at once. “Don’t block the event loop” is Node’s prime directive. Real CPU work has to move to worker_threads or a cluster of one process per core. Brilliant for I/O; awkward for compute.
The loop has phases. Each turn runs in a fixed order: timers → pending callbacks → poll (where I/O events fire) → check (setImmediate) → close. Understanding the order is how you reason about why a setTimeout(fn, 0) and a setImmediate(fn) can resolve in surprising sequence.
Microtasks jump the queue. Between every phase, Node fully drains the microtask queues — process.nextTick first, then Promise jobs. A tight recursive nextTick can starve the loop’s I/O phases entirely, a subtle cousin of blocking the loop.
Three ways to use more cores. worker_threads share memory via SharedArrayBuffer (good for CPU work in-process); cluster forks one process per core behind a shared socket (good for scaling the whole server); child_process shells out. None of them change the rule that any one loop must never block.
Not the browser’s loop. The browser event loop has one macrotask per turn and a rendering step; Node’s is libuv’s phased loop with no rendering. Same mental model, different rulebook — a frequent source of “works in the browser, not in Node” timing bugs.
Go is the other answer to the burst — not “never block a thread” but “make the thread so cheap you stop counting.” It is the green-thread model turned into a language keyword, scheduler and all. That is why Go became the default for network servers.
Go builds the cheap thread into the language. A is any function with the word go in front of it, and it starts on a 2 KB stack that grows and shrinks on demand — against 8 MB for a pthread. Launching one is nearly free; a million is a normal afternoon. Where Java spent 27 years arriving at cheap threads, Go shipped with them in 2009.
Cheap threads still need scheduling, and Go does it in user space with the : G is a goroutine, M is a machine (an ), P is a processor — a per-core run queue. Millions of Gs ride a handful of Ms, and (defaulting to your core count) sets how many run at once. It is the same idea as Java’s virtual threads, baked into the runtime from day one.
Now the part that matters. You write conn.Read(buf) as if it blocks — plain, synchronous, readable. But the runtime intercepts the syscall, parks the goroutine, hands its M to another goroutine, and resumes yours when the I/O is ready, via a built on epoll/kqueue. You get blocking code and efficiency at the same time — no callbacks, no coloring, no catch.
Because the scheduler lives in user space, a goroutine switch is about 170 ns and never enters the kernel — against ~1–2 µs for an OS . Scheduling is cooperative at function calls, with async preemption since Go 1.14 so a tight loop cannot hog a P. An order of magnitude cheaper per switch is exactly what lets one box juggle hundreds of thousands of goroutines without the scheduler catching fire.
Goroutines coordinate with , not shared locks. The proverb comes straight from Hoare’s 1978 CSP: “Do not communicate by sharing memory; instead, share memory by communicating.” A channel hands a value from one goroutine to another; select waits on several at once. Concurrency stops being a pile of locks and becomes a dataflow you can read.
Work-stealing keeps cores busy. Each P has its own run queue; when a P empties it steals half of another P’s queue (or pulls from the global queue). That is how a lopsided burst — all the goroutines landing on one P — still spreads across every core without a central bottleneck.
sysmon and preemption. A background monitor thread,sysmon, watches for goroutines that have run too long and, since Go 1.14, signals an async preemption so they yield. Before 1.14 a tight loop with no function calls could stall a whole P — a classic old-Go hang.
The netpoller. Blocking network calls do not block an M; they register the fd with epoll/kqueue and park the goroutine. A dedicated poller wakes parked goroutines when their fd is ready — the same kernel primitive as Node’s loop, hidden behind synchronous-looking code.
Channels are not free. An unbuffered channel is a synchronization point — sender blocks until a receiver is ready — while a buffered channel decouples them up to its capacity. Misjudge that and you get goroutine leaks: senders parked forever on a channel nobody reads. select with a context cancel is the usual cure.
Python has the richest concurrency story precisely because of its most argued-about feature — the GIL. It shapes every answer: threads that help only I/O, processes that cost memory, an event loop borrowed from Node, and, as of October 2025, two brand-new routes to real parallelism.
Python has real OS threads — and a lock that makes them take turns. The lets exactly one thread execute Python bytecode at a time, no matter how many cores you own. It has guarded CPython’s reference-counting memory management since around 1992: simple, fast for single threads, and the reason a 64-core box can leave 63 cores idle while one thread runs Python.
The saving grace: a thread releases the GIL while blocked on I/O. So threading.Thread genuinely helps work — 10,000 threads can wait on sockets concurrently, taking turns cleanly. But CPU-bound work is serialized: the GIL is held during computation, so two threads crunching numbers run slower than one, burning cycles fighting over the lock. Threads for waiting, never for math.
To actually use every core, skip threads and fork processes. gives each process its own interpreter and its own GIL — genuine parallelism. The bill: no shared memory, so data is pickled across a pipe, and each process costs megabytes. Wonderful for CPU crunching across 8 cores; far too heavy to hold 10,000 idle connections.
For 10,000 I/O-bound clients, Python’s real answer is asyncio (2014): one thread, one , driven by — the same trick as Node. No GIL fight (one thread), no thread stacks (coroutines are cheap objects). The cost is a separate, colored world: it needs async libraries top to bottom (asyncpg, not psycopg2).
Python 3.14 (October 2025) added a fourth option: (PEP 734), exposed through the new concurrent.interpreters module and InterpreterPoolExecutor. Many interpreters live inside one process, each with its own GIL, so they run bytecode in true parallel while sharing no globals — passing data over queues. “The isolation of processes with the efficiency of threads,” and a native home at last for the and actor models.
The other 3.14 headline: PEP 779 makes the (no-GIL) build officially supported, graduating the 3.13 experiment (PEP 703, by Sam Gross at Meta). One interpreter, many threads, real parallelism, no lock — for about 5–10% single-thread overhead in 3.14, down from ~40% in 3.13’s first cut. After more than thirty years, the GIL is finally a build option, not a law.
Why the GIL survived so long. CPython manages memory by reference counting, and a lock-free refcount is either slow or wrong on multiple cores. The GIL made every refcount safe for free — and the entire C-extension ecosystem (NumPy, pandas, thousands more) relied on that guarantee without ever saying so, which is why removing it risked breaking the world.
The failed and the winning attempts. Larry Hastings’ “Gilectomy” (2016) removed the GIL but made single-threaded code far slower. PEP 703 succeeded where it failed by using biased reference counting and per-object locks, keeping the single-thread penalty small enough to ship.
Two bets on the future. 3.14 shipped both routes at once. PEP 684’s per-interpreter GIL (3.12) laid the groundwork for PEP 734’s stdlib subinterpreters — isolation with parallelism — while PEP 779 free-threading removes the lock entirely for shared-memory parallelism. Which one wins in practice is the open question of Python’s next few years.
One burst of 10,000 clients, four runtimes, one board. Read across and the whole tutorial collapses to a single shape: a few OS threads multiplexing thousands of logical tasks — only the multiplexer changes.