Effect HttpApi cold start

effect npm 4.0.5 · node v24.21.0 · linux 6.18.54 · Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (8 cores) · rolldown 1.2.14 · 30 fresh processes per size · Sat, 17 Oct 2026 09:53:24 GMT

Cold start = start the server and answer one request. Each sample launches a brand-new node process running a single rolldown bundle of an effect/http-api server (HttpRouter.serve on @effect/platform-node's NodeHttpServer), waits until it is listening, sends one real HTTP GET over loopback and stops the clock when the response has been fully received. The clock starts just before spawn(), so it covers everything: process creation, Node boot, parsing, compiling and evaluating the bundle, defining the API, building the handler, binding the socket, and serving a request with code V8 has never run before. APIs have 1, 10, 50, 100, 500 endpoints.

1 · Anatomy of a cold start

Every phase from spawn() to the HTTP response arriving back at the client, as medians and in execution order. Spawn is process creation plus the time for the "listening" line to reach the client (measured with an empty script: 37.9 ms to its first output line, which also includes Node boot). Define API is HttpApi.make(…).add(…) with all its schemas. Build handler + listen is HttpApiBuilder plus building the HttpRouter.serve layer on NodeHttpServer until the socket is bound. First HTTP request is the client's round trip for one GET, of which 6.2 ms is spent inside the server.

• Before any API code runs, a server has already spent 113 ms on spawning, booting Node and loading the 765 KB bundle: 81% of a small server's cold start.
• Define + build + listen grow with the API: 19 ms at 1 endpoint, 90 ms at 500.
• The first HTTP request takes 7.6–10.1 ms. It is cold too: the server runs the request path (parsing, routing, decoding, encoding) for the first time.

Web-handler mode (Worker-style)

The same app as a web handler (HttpRouter.toWebHandler, platform-neutral bundle, request passed in-process as a Request), which is how it would run on Cloudflare Workers or another fetch-based runtime. There is no spawn, socket or Node server stack, and the clock runs from process start to the response body. This mode is also what the CPU profiles and the cold/warm experiment below use.

• Small APIs are dominated by fixed cost. At 1 endpoint, 85% of the cold start happens before any API code runs. Of the 57 ms module load, ~17 ms is parsing and compiling the 552 KB bundle and ~27 ms is Node initializing undici (Node-only, see §4).
• In large APIs, define + build become the largest part: 46% of the cold start at 500 endpoints, ahead of module load (34%) and Node boot (15%).
• The first request always pays ~8 ms regardless of size. A warm request takes 0.6 ms, so this is almost entirely first-time code execution (see §2).
• CPU exceeds wall time. V8's concurrent compiler and GC threads add CPU on top of the main thread: 246 ms CPU vs 179 ms wall at 500 endpoints. That matters wherever startup CPU is capped or billed.

2 · Cold code vs real work

To separate the cost of running code for the first time from the work itself, each process runs the cold pass and then rebuilds everything from scratch five more times in the same process: a new API definition, layer, handler and request each time. We report the last rebuild. The work is identical, but V8 has by then lazily compiled the functions, tiered them up from the interpreter and warmed inline caches, and any one-time lazy initialization has happened. Cold − warm is the cold-code penalty, a cost that exists only because it is a cold start.

• At 1 endpoint, 94% is penalty: the work takes 1.0 ms hot but 15.5 ms cold. The first request alone is 17× slower cold (7.5 vs 0.4 ms), and the handler build 17×.
• The penalty grows with API size too (47 ms at 500 endpoints) but its share shrinks to 53%: later endpoints reuse code paths the first ones already warmed. That is why the per-endpoint cost is higher for small APIs (396 µs/ep from 1→10) and settles at ~138 µs/ep.
• Implication. Algorithmic tuning of define/build only addresses the warm 82 µs/ep. Cutting the amount of distinct code executed at startup (fewer layers of indirection, fewer generic combinators on the build path, deferring per-route setup until a route is first hit) is what moves cold start.

3 · How cold start grows with API size

Cold start grows linearly: ~154 µs per endpoint on top of ~103 ms fixed (R² 0.996). Define and build share the per-endpoint cost about evenly. Module load grows only with bundle size (+0.4 KB, +6.9 µs per endpoint), and the first request is flat.

What the per-endpoint cold cost is spent on

CPU profiles (5 per size) are split into phases using monotonic timestamps from the harness, and samples are attributed to Effect source files via sourcemaps. Category times are profile shares scaled to the un-profiled medians, and the per-endpoint figure is each category's linear-fit slope.

• Define (67 µs/ep): mostly Schema construction (23 µs: schema/make.ts, SchemaAST.struct), then HttpApi bookkeeping (12 µs, including validateResponseExclusivity on every endpoint) and pipe/dual (9 µs).
• Build (77 µs/ep): the router is the largest single part (22 µs: FindMyWay._on does string slicing, toLowerCase and split/join per segment), then per-route setup in HttpApiBuilder (13 µs: handlerToRoute, encoders, decoders), the Effect runtime running the layer (8 µs) and Schema codec derivation (13 µs).
• GC adds 14 µs/ep: startup allocates many short-lived AST and closure objects.

4 · Module load: the fixed part

Module load takes ~57 ms at every size, before a single line of API code runs.

• Parse & compile (~30%) scales with bundle size. HttpApi pulls in Schema, the HTTP server stack (request/response, cookies, multipart), streams and the router whatever the API uses, so the floor is 552 KB unminified.
• undici init (~48%, Node only). Node creates its fetch-related globals lazily, and the first touch loads undici and its dependencies (crypto, zlib, web streams). workerd has these natively. See below for what triggers it.
• Effect's own module evaluation (services, tags, schema singletons) is ~10 ms.

Measured levers

• Minify: −13 ms on average (552 → 220 KB). Works on any runtime.
• Node compile cache (NODE_COMPILE_CACHE, V8 code cache on disk): −16 ms of module load. It leaves define/build unchanged, because the cache holds the code compiled at load time, while the functions that only run later during define and build are still compiled lazily on first call.

Why every Node server pays for undici

Two independent things load undici during startup, each costing about 30 ms in isolation (Node v24.21.0):

1. Schema.ts evaluates instanceOf(globalThis.FormData) at module scope. Touching FormData in an empty process costs 29.2 ms.
2. An ESM import … from "node:http", which @effect/platform-node does. Node 24's http module exposes lazy, undici-backed WebSocket, CloseEvent and MessageEvent getters, and an ESM import snapshots every export, which triggers them: import("node:http") costs 32.2 ms, while process.getBuiltinModule("node:http") (or require) costs 4.7 ms.

Removing either trigger alone changes nothing, because the other one still loads undici. Removing both cuts server cold start by 17 ms (157.7 → 140.6 ms), and undici is never loaded. In web-handler mode the FormData reference is the only trigger: removing it drops module load from 58 to 28 ms, but the cost reappears in the first new Request() (7.5 → 34.4 ms), so on Node the total doesn't change; on workerd these globals are native and the cost doesn't exist.

5 · Where to cut cold start

1. Defer per-route setup to the first hit on that route. Build is 77 µs/endpoint and is paid upfront for every route. Compiling a route's codecs and handler pipeline (handlerToRoute, SchemaAST.copy/make) on first request would make cold start pay only for the routes a fresh instance actually serves.
2. Execute less distinct code at startup. 94% (small) to 53% (large) of post-load time is cold-code penalty. Fewer generic layers on the define/build path (dual/pipe wrappers, Effect-runtime-driven layer construction for what is synchronous setup) cut both the work and the code that has to be compiled.
3. Cheaper route registration. FindMyWay._on is the top self-time function in build. Its per-segment split/join and toLowerCase could be skipped for plain segments, or the tree could be built lazily on first lookup.
4. Stop loading undici in Node servers (−17 ms per cold start, measured). Two changes are needed together: make the FormData/File guards in Schema.ts read the global lazily inside the check, and have @effect/platform-node obtain node:http through process.getBuiltinModule("node:http") (Node ≥ 20.16 / 22.3) instead of a static ESM import.
5. For users: minify (−13 ms), enable the compile cache where the platform allows (−16 ms on Node), and reuse schema values across endpoints.

Methodology

• Server mode (headline). Harness + generated API + @effect/platform-node bundled by rolldown (platform: "node"). The runner, a long-lived Node process, records the time, spawns the server, waits for its "listening" line (port 0 on 127.0.0.1), sends GET /r0/1 (/r0 for 1-endpoint groups) with a fresh connection, and stops the clock when the response body has been read. The server reports its own phase times and its CPU when the response is sent.
• Web-handler mode. Harness + generated API bundled by rolldown into one ESM file (platform: "neutral", JSDoc stripped, unminified unless stated). effect resolves from the published npm package exactly like an app. Each sample is a new node process: two discarded warmup processes per size (to warm the OS file cache only; every measured process is a cold V8 isolate), then 30 measured, run sequentially. Node's compile cache is off except in the lever experiment.
• Cold start = time from process start (performance.timeOrigin) until the first response body has been read. Node boot is the median time-to-first-line of an empty script (27.3 ms). CPU is process.cpuUsage() at the first response (all threads).
• API shape. G groups × E endpoints (1×1, 1×10, 5×10, 10×10, 50×10). Each group is a REST resource with an entity struct (int, strings, literal union, optional, array, nested struct), create/update payloads, a TaggedError 404, a struct 409, and list/get/create/update/delete endpoints plus actionN endpoints with their own schemas. Handlers are trivial.
• Cold vs warm. In-process rebuilds with BENCH_WARM=5, last iteration reported, 15 processes per size.
• Attribution. --cpu-prof at 100 µs, 5 profiles per size. Samples are assigned to phases with process.hrtime marks (the profiler's own monotonic clock) and to sources through sourcemaps chained into Effect's .ts files. Before the harness runs, samples are classified as bundle compile (under compileSourceTextModule), module code (bundle frame on the stack) or Node boot. node: frames under an undici frame count as undici init.
• Caveats. Node is not workerd. V8 is the same, so the scaling, the cold-code penalty and the hot spots carry over, but absolute times, Node boot and the undici cost do not.

Appendix

Not part of cold start: OpenAPI generation. OpenApi.fromApi costs 151 µs/endpoint (84 ms at 500 endpoints), but HttpApiScalar, HttpApiSwagger and openapiPath all build it lazily and memoize it, so only the first /docs request on an instance pays it.

Reproduce: node scripts/collect.mjs && node scripts/experiment-undici.mjs && node scripts/render-report.mjs · source: github.com/mikearnaldi/benchmark-http-api-boot