Skip to content

perf: cut map()/sourceAndMap() allocations via mappings writer and streaming ReplaceSource#251

Merged
alexander-akait merged 2 commits into
mainfrom
claude/webpack-sources-perf-x55foa
Jul 5, 2026
Merged

perf: cut map()/sourceAndMap() allocations via mappings writer and streaming ReplaceSource#251
alexander-akait merged 2 commits into
mainfrom
claude/webpack-sources-perf-x55foa

Conversation

@alexander-akait

Copy link
Copy Markdown
Member

Summary

Deep-profiled a webpack-like chunk pipeline (ReplaceSource-wrapped OriginalSources + SourceMapSources assembled by ConcatSource under CachedSource) with the V8 CPU profiler and sampling heap profiler. Three sites accounted for ~55% of all allocations; this PR addresses each without changing any output:

  • Byte-buffer mappings writer. createMappingsSerializer built every mapping out of ~5 intermediate string concatenations and all three internal callers appended the result with mappings += str (31% of workload allocations). The full serializer now writes char codes into a growing Uint8Array via a new internal createMappingsWriter and materialises the mappings string once in finish(). The lines-only variant keeps the string fast paths (constant ";AACA" segments + cons append measured faster there) behind the same writer interface. The public string-returning createMappingsSerializer API is unchanged.
  • Line-offset index in ReplaceSource.checkOriginalContent. Instead of lazily splitting each source content into an array of line substrings (splitIntoLines, 11% of allocations), it builds an exact-size Uint32Array of line-start offsets and compares with String.startsWith — no per-line strings, no comparison slices.
  • ReplaceSource.streamChunks honors finalSource like OriginalSource/RawSource already do: it emits position-only chunks (chunk === undefined) and returns the whole replaced source once, assembled in O(replacements) string operations by source(). This removes the per-token code += chunk cons chain in every enclosing consumer (ConcatSource, getSourceAndMap, streamAndGetSourceAndMap); all in-tree consumers already handle undefined chunks under finalSource.

Measured (interleaved A/B vs main, medians): workload total allocations 16.7 → 6.6 MB/iter (−60%), wall time −14%. Isolated paths: ConcatSource.sourceAndMap −16…−20% time (alloc 279 → 130 KB), ReplaceSource.sourceAndMap −8% (63 → 37 KB), ReplaceSource.map({columns:true}) −8%, OriginalSource.map({columns:true}) allocations 37 → 9 KB/op at equal time, columns:false paths at parity. Sources and maps are byte-identical to main across the fuzz suite.

Rejected after measurement: a byte-writer for lines-only mappings (+50% CPU → hybrid instead); skipping source() when options.source === false (unsafe — CachedSource forwards those options and caches the returned source). Known trade-off: standalone ReplaceSource.map({columns:false}) is ~+12% (~0.03 ms) because returning the assembled source is required for CachedSource correctness; webpack always consumes ReplaceSource through Concat/Cached wrappers, which all got faster.

What kind of change does this PR introduce?

Performance improvement (patch, changeset included). No behavior change.

Did you add tests for your changes?

No new tests needed — the change is output-invariant and covered by the existing suites: all 89,888 tests pass (including the fuzz suite and 1373 snapshots), plus lint/types are green.

Does this PR introduce a breaking change?

No. The public createMappingsSerializer API is unchanged; createMappingsWriter is a new internal export. ReplaceSource.streamChunks emitting undefined chunks under finalSource: true follows the documented contract already used by OriginalSource and RawSource.

If relevant, what needs to be documented once your changes are merged or what have you already documented?

Nothing — internal performance change; a changeset describes it for the release notes.

Use of AI

This PR was written with Claude Code (profiling, implementation, and A/B measurement).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KDFBEvi3ctDSyHQ7f9izX5


Generated by Claude Code

…reaming ReplaceSource

Deep-profiled a webpack-like chunk pipeline (ReplaceSource-wrapped
OriginalSources + SourceMapSources assembled by ConcatSource under
CachedSource) with the V8 CPU profiler and sampling heap profiler.
Three hot spots accounted for ~55% of all allocations; this addresses
each without changing any output:

- createMappingsSerializer built every mapping out of ~5 intermediate
  string concatenations and callers appended the result with
  `mappings += str` (31% of workload allocations). The full serializer
  now writes char codes into a growing Uint8Array via
  createMappingsWriter and materialises the mappings string once in
  finish(). The lines-only variant keeps the string fast paths
  (constant ";AACA" segments + cons append measured faster there) behind
  the same writer interface. The public string-returning
  createMappingsSerializer API is unchanged.

- ReplaceSource.checkOriginalContent lazily split each source content
  into an array of line substrings (splitIntoLines, 11% of
  allocations). It now builds a Uint32Array of line-start offsets
  (exact-size, two-pass scan) and compares with String.startsWith —
  no per-line strings, no comparison slices.

- ReplaceSource.streamChunks now honors finalSource like
  OriginalSource/RawSource: it emits position-only chunks
  (chunk === undefined) and returns the whole replaced source once,
  assembled in O(replacements) string operations by source(). This
  removes the per-token `code += chunk` cons chain in every enclosing
  consumer (ConcatSource, getSourceAndMap, streamAndGetSourceAndMap);
  all in-tree consumers already handle undefined chunks under
  finalSource.

Measured on the profiling workload (interleaved A/B vs main, medians):
total allocations 16.7 -> 6.6 MB/iter (-60%), wall time -14%.
Isolated paths: ConcatSource.sourceAndMap -16..-20% time,
ReplaceSource.sourceAndMap -8%, ReplaceSource.map(columns:true) -8%,
OriginalSource.map allocations 37 -> 9 KB/op at equal time,
columns:false paths at parity. All 89,888 tests pass; sources and maps
are byte-identical to main across the fuzz suite.

Claude-Session: https://claude.ai/code/session_01KDFBEvi3ctDSyHQ7f9izX5
Copilot AI review requested due to automatic review settings July 5, 2026 15:18
@changeset-bot

changeset-bot Bot commented Jul 5, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 77d2620

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
webpack-sources Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@linux-foundation-easycla

linux-foundation-easycla Bot commented Jul 5, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.29%. Comparing base (bf3033c) to head (77d2620).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #251      +/-   ##
==========================================
+ Coverage   98.14%   98.29%   +0.14%     
==========================================
  Files          25       25              
  Lines        2102     2283     +181     
  Branches      678      731      +53     
==========================================
+ Hits         2063     2244     +181     
  Misses         37       37              
  Partials        2        2              
Flag Coverage Δ
integration 98.29% <100.00%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Jul 5, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 23.35%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 13 improved benchmarks
❌ 15 regressed benchmarks
✅ 184 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory original-source memory: sourceAndMap({ columns: true }) 2 KB 967.2 KB -99.8%
Memory prefix-source memory: new PrefixSource() 1.5 KB 169.7 KB -99.1%
Memory raw-source memory: new RawSource(string) 784 B 13,992 B -94.4%
Memory webpack-20961: warm + full clearCache() per chunk (most aggressive) 2.3 MB 11.3 MB -79.2%
Memory webpack-20961: warm + clearCache({maps,source:false,parsedMap}) per chunk (PR #20963) 2.4 MB 11.3 MB -78.67%
Memory webpack-20961: warm CHUNKS × MODULES, hold all live (baseline) 2.5 MB 10.1 MB -75.26%
Memory concat-source memory: source() concatenates children 115.2 KB 196 KB -41.23%
Memory original-source memory: map({ columns: true }) builds full mappings 541.4 KB 903.7 KB -40.09%
Memory replace-source memory: map({ columns: true }) splices mappings 2.7 MB 4.2 MB -35.43%
Simulation helpers/createMappingsSerializer: full serializer 3.7 ms 5.2 ms -27.31%
Memory concat-source memory: new ConcatSource(...children) 3.1 KB 4 KB -22.24%
Memory compat-source memory: CompatSource.from() short-circuits on Source 520 B 648 B -19.75%
Memory raw-source memory: new RawSource(buffer) 784 B 976 B -19.67%
Memory concat-source memory: map({ columns: true }) composes child maps 3.3 MB 4 MB -15.63%
Memory clear-cache memory: unique tasks (clearCache default) 3.2 MB 3.8 MB -14.83%
Memory clear-cache memory: shared modules (visited set — single allocation) 198,704 B 776 B ×260
Memory clear-cache memory: shared modules (no visited set — allocates per chunk) 408.7 KB 19.7 KB ×21
Memory compat-source memory: new CompatSource(sourceLike) 2,800 B 832 B ×3.4
Memory compat-source memory: delegated source() + map() through wrapper 22.3 KB 9.4 KB ×2.4
Memory size-only-source memory: new SizeOnlySource() 99.1 KB 58.8 KB +68.51%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/webpack-sources-perf-x55foa (77d2620) with main (bf3033c)

Open in CodSpeed

…sions

Addresses the three CI failures on the previous commit:

- lint: reformat the line-start scan loops for prettier 3.9.4 (the
  version pinned by package-lock; a stale local install accepted the
  old shape).

- codecov/project: the legacy lines-only serializer and several writer
  branches lost integration coverage after the writer refactor. Add
  unit tests asserting writer/serializer output equivalence over
  branch-covering and buffer-growth event streams, plus exact-encoding
  cases. createMappingsSerializer.js is back to 100% line coverage and
  overall lib coverage now exceeds the main baseline.

- CodSpeed simulation regressions (benchmarks run under --no-opt, where
  call indirection and extra passes cost far more than JIT-mode wall
  time showed):
  - createLinesOnlyMappingsWriter now inlines the encoding instead of
    delegating to the string serializer per line
    (original-source map({columns:false}) back to parity).
  - checkOriginalContent builds line starts in a single pass with a
    plain array (replace-source streamChunks back below base).
  - ReplaceSource.streamChunks skips assembling the final source when
    the caller passed source: false (getMap does); getSourceAndMap and
    streamAndGetSourceAndMap explicitly override the flag because they
    consume/cache the text — the streamChunks fallback forwards getMap
    options into sourceAndMap, which the fuzz suite caught.

Interpreter-mode A/B vs main after these fixes (medians): replace-source
map() -12%, sourceAndMap() -32%, streamChunks(1000 replacements) -3%,
original-source map({columns:false}) parity, pipeline streamChunks
(columns off) -6%, full-serializer benchmark at parity (its flagged -27%
compares byte-identical code — environment skew, matching the report's
"different runtime environments" warning). Local ground truth for the
flagged webpack-20961 memory rows: allocations 160 MB -> 7.3 MB and
retained heap 42.4 MB -> 3.3 MB in favor of this branch (the writer
produces one flat mappings string instead of a huge retained cons tree).

All 89,895 tests pass; lint and types are green.

Claude-Session: https://claude.ai/code/session_01KDFBEvi3ctDSyHQ7f9izX5
@alexander-akait alexander-akait force-pushed the claude/webpack-sources-perf-x55foa branch 2 times, most recently from 2266774 to 77d2620 Compare July 5, 2026 15:54

Copy link
Copy Markdown
Member Author

/easycla


Generated by Claude Code

Copy link
Copy Markdown
Member Author

Triage of the remaining CodSpeed warnings (all Memory mode; no Simulation/instruction-count regressions remain as of the latest runs):

The flagged memory rows are measurement artifacts, not effects of this PR:

  1. Same-tree flip-flops. e88ad27 and 2266774 contain byte-identical code (the latter is a commit-message-only rewrite), yet between their two runs several rows reversed direction: concat-source map({columns:true}) ⚡ +33.65% → ❌ −11.5%, size-only-source new SizeOnlySource() ❌ −18.94% → ⚡ +71.84%, raw-source new RawSource(string) ✅ → ❌ −94.53%.
  2. Untouched code. RawSource, SizeOnlySource, ConcatSource constructor, CompatSource delegation and ReplaceSource.source() are not modified by this diff.
  3. Physically impossible baselines. original-source sourceAndMap({columns:true}) reports a 2 KB base, but a single call must allocate more than that for its returned source + mappings alone.
  4. Ground truth for the headline webpack-20961 rows (measured with the V8 sampling heap profiler on the exact benchmark shape, includeObjectsCollectedBy{Major,Minor}GC, interleaved base/head): total allocations 160 MB → 7.3 MB (full clearCache() variant) and retained heap after GC 42.4 MB → 3.3 MB — in favor of this branch. The old mappings += str path retains large cons-string trees; the writer produces one flat string.

Happy to share the measurement scripts if useful. Given the above, the memory-mode warnings look acknowledgeable on the CodSpeed dashboard.


Generated by Claude Code

@alexander-akait alexander-akait merged commit f38af31 into main Jul 5, 2026
34 of 35 checks passed
@alexander-akait alexander-akait deleted the claude/webpack-sources-perf-x55foa branch July 5, 2026 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants