Skip to content

perf(bundler): a source's symbols the pack planner treats alike ship as one module - #352

Merged
thejackshelton merged 3 commits into
mainfrom
perf/group-symbol-modules
Oct 29, 2026
Merged

thejackshelton merged 3 commits into
mainfrom
perf/group-symbol-modules

Conversation

@thejackshelton

@thejackshelton thejackshelton commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

TLDR: a packed client build now ships a source's symbols as one module when the pack planner cannot tell them apart. On a 40-page cut of the shadcn port, client modules fall from 7,845 to 4,787. Every page downloads less JavaScript at load (4.2 MB raw and 513 KB gzip less across the 40 pages), and executes the same or less. Lever 4 of #268. It re-anchors G16 (#360) and the build-scale module count (#353) down in the same change.

Why

Rolldown spends about 90 KB of native memory, plus real time, on every module however small (repro: 40,000 one-function modules take 3.75 GB and 6.9 s; the same code in 1,000 modules takes 355 MB and 0.3 s). The full shadcn port has 80,209 client modules, and 40,211 of them are one-function symbol modules. That is what keeps the build off 16 GB runners.

What merges

In packages/bundler/src/build/symbol-groups.ts, symbols of one source share a module only when all of these match:

  • what the file's own demand map says about them: kind, whether resume start runs them, whether some action's first use names them, and which payload record kinds list them;
  • the modules they import, so loading a group sets up nothing a member would not.

Only members that are pure function declarations merge, so loading a group runs no other member's top-level code. Derives and initializers, which render data imports statically, keep their own modules. Builds that log execution keep one module per symbol. A group is named by a hash of its members, so two compiles of one file that group differently never share a module id.

When a page needs only part of a group, it loads the whole group

The page-wide planner can need only some members of a group: a shared component whose pages use different controls, or another file's first use naming one member. On the docs site this happened 245 times. The first version of this PR left such a group to load on first call, and /ui/accordion then fetched JS on a first click, which perf guard G2 forbids.

Owner ruling: the page loads the whole group with itself. This relaxes the earlier "same code per page" rule: a page may now also load the unneeded members of a group it partly needs. In exchange, two rules hold per page, pinned by perf guard G16 (#360):

  • JavaScript executed at load must not increase (lowest of three loads; 256 chars of measured swing allowed);
  • JavaScript downloaded at load must stay at or below main.

G2 (no JS fetched on a first click) still holds. All guards on this branch with G16 applied: every guard holds, and G16 reads lower on all six guarded pages:

page executed at load (main → this) downloaded gzip (main → this)
bench / 10.8 → 10.8 KB 85.1 → 82.9 KB
bench /records 10.8 → 10.8 KB 85.1 → 82.9 KB
bench /settings 10.7 → 10.8 KB (inside the swing) 85.1 → 82.9 KB
docs / 143.5 → 139.9 KB 1,518 → 1,499 KB
docs /concepts/state 145.1 → 141.5 KB 1,625 → 1,605 KB
docs /ui/accordion 46.7 → 46.5 KB 398.5 → 381.0 KB

On the 40-page shadcn cut, loading whole groups still downloads less than main on every page. Executed bytes match main, apart from the ~8 KB swing a few pages show between runs of either build.

One runtime change it needed

Loading a symbol ran for (const name in mod) over the module namespace to find chunk initializers. Through the packed namespace proxy, that runs every export's getter. With groups it ran the other members' getters at load: about 32 bytes each, visible as +100 to +300 bytes executed per page. The resolver now walks Reflect.ownKeys(mod), which reads no export. packages/compiler/test/symbol-resolver-emit.test.ts pins that only the initializer is read. The test fails on the old scan. The compiler's byte-equality snapshot changes in exactly those two lines.

Checks, per page, in Chromium (40-page cut, min of three runs)

  • JS downloaded at load: lower on all 40 pages.
  • JS executed at load: lower on 39 pages. On the 40th, the old build swings by 8 KB between runs (102 KB in one run, 110 KB in two), and the new build sits inside that swing at 110 KB.
  • Same code per page: the symbols each page loads at load are the same as with perf(bundler): initializer symbols load without a re-export module in front #348 alone. Groups were expanded to their members via byte-attribution.json.
  • Clicking every tab on eight docs pages switches panels the same way on both builds, with no new console errors.

Build cost (40-page cut, 3 workers, two paired cold runs on a busy machine)

before after
client / ssr / nitro modules 7,845 / 2,037 / 947 4,787 / 2,037 / 947
process peak footprint 8.4, 8.3 GB 8.0, 6.6 GB
V8 memory peak 3.6, 3.6 GB 2.9, 3.0 GB
Rolldown native peak 2.36, 2.50 GB 2.20, 2.29 GB
build CPU 128, 134 s 144, 114 s (no clear change at this size)

Full-port numbers will follow on #268 once the build machine has room.

Tests

pnpm run typecheck and the fast local CI run pass (6,000+ tests). dense-async-symbol-table.test.ts now reads the manifest a packed build writes, so it checks each symbol's grouped module.

Note

Ship demand-equivalent symbols as one grouped module in the bundler

  • Groups symbols the pack planner treats alike (same kind, resume, first-use, and record-kind attributes, plus the same imported-module sources) and emits their sources as a single shared virtual module in symbol-groups.ts.
  • The transform pipeline (transform.ts) only does this when experimental native packing is enabled and execution-log module hooks are off. Ungrouped symbols keep individual virtual modules.
  • Route closures (interaction-closures.ts) seed the grouped module whenever a demanded symbol belongs to it, so group modules land in the right boot or first-use closure.
  • Members are excluded from a group when their sources cannot be parsed, contain conflicting local names, or use unsupported declarations (only named function declarations merge).
  • Also changes the generated symbol-resolver initializer scanner to use Reflect.ownKeys and read only matching string-named initializer exports, instead of for-in enumeration over module exports.
  • Risk: Behavioral Change — grouped symbols no longer get individual virtual modules; check manifest entries and compiler symbol rows in transform.ts, and consumers of runtimeDemandSources in rolldown.ts now receive a symbolGroups map. Build-scale budgets drop from 14911 to 4495.

Macroscope summarized edbdbc1.

Anchors this change moves (CI, linux-x64)

main this
build-scale client modules at 180 pages 14,911 4,495
client modules per extra page 79.7 22.5
G16 downloaded at load, bench pages 88,497 B 87,007 B
G16 docs / (executed, downloaded) 113,031 chars, 1,579,694 B 110,779 chars, 1,564,841 B
G16 docs /concepts/state 124,826 chars, 1,693,361 B 121,829 chars, 1,678,055 B
G16 docs /ui/accordion downloaded 417,354 B 406,069 B

Bench pages run 1 char more at load (11,370 to 11,371 on /), inside G16's swing; their executed anchors stay at main's value. The darwin-arm64 G16 anchors are refreshed in a follow-up measured on a Mac.

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 29, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Approvability

Verdict: Approved at edbdbc1

Macroscope's review found this PR approvable — Packed client builds combine eligible symbols into shared modules to reduce bundler overhead. Conservative grouping restrictions, group-aware loading and reported browser measurements limit the performance and execution risks.

Notes:

  • All code in this push has already been reviewed. Approvability was decided on eligibility alone.

You can add or adjust custom eligibility rules. Learn more.

@thejackshelton
thejackshelton force-pushed the perf/group-symbol-modules branch from c362023 to f3cb112 Compare October 29, 2026 04:00
@macroscopeapp
macroscopeapp Bot dismissed their stale review October 29, 2026 04:00

Dismissing prior approval to re-evaluate f3cb112

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 29, 2026
…as one module

Rolldown spends about 90 KB of native memory and real time on every
module, however small, and the shadcn port's client graph is mostly
one-function symbol modules (40,211 of 80,209 modules). A packed client
build now merges a source's symbols into one module when its own demand
map cannot tell them apart: same kind, same resume use, named or not by
an action's first use, same payload record kinds, and the same imported
modules. Only function-declaration members merge, so loading a group runs
no other member's code; derives and initializers, which render data
imports statically, keep their own modules. Builds that log execution
keep one module per symbol.

The page-wide planner can still see members differently than the file's
own map does (another file's first use naming one of them). The closure
step guards that: a group counts as first-use code on a route only when
the route names every member before input, or a walk took it whole;
otherwise it leaves the route's first-use closures and its members load
on first call, never earlier.

Loading a symbol scanned its module's namespace with for...in, which
runs every export's getter through the packed namespace proxy; with
grouped modules that ran other members' getters at load. The resolver
now walks Reflect.ownKeys, which reads no export.

On a 40-page cut of the shadcn port: client modules 7,845 to 4,787. In
Chromium on all 40 pages, JavaScript downloaded at load falls on every
page (4.2 MB raw, 513 KB gzip in total) and JavaScript executed at load
falls on 39 pages; the 40th moves inside an 8 KB swing its old build
also shows (min of three runs). The modules every page loads at load are
the same symbols as before, and clicking every tab on eight pages works
as before.
…e group

A group is shared by symbols the file's own demand map cannot tell apart,
but a page can still need only some of them (a shared component whose
pages use different controls). The first version left such a group to
load on first call, and the docs site's /ui/accordion then fetched JS on
a first click (perf guard G2). Owner ruling: the page loads the whole
group with itself instead. What a page runs at load must not grow and
what it downloads must stay at or below main; G15 (#360) holds both.
@thejackshelton
thejackshelton force-pushed the perf/group-symbol-modules branch from f3cb112 to d712f96 Compare October 29, 2026 05:10
@macroscopeapp
macroscopeapp Bot dismissed their stale review October 29, 2026 05:10

Dismissing prior approval to re-evaluate d712f96

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 29, 2026
…mbol modules (linux-x64 G16 from CI run 40815323579; build-scale 4,495 client modules at 180 pages)
@macroscopeapp
macroscopeapp Bot dismissed their stale review October 29, 2026 05:16

Dismissing prior approval to re-evaluate edbdbc1

@thejackshelton
thejackshelton added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit b7cebce Oct 29, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant