perf: cut map()/sourceAndMap() allocations via mappings writer and streaming ReplaceSource#251
Conversation
…reaming ReplaceSource Deep-profiled a webpack-like chunk pipeline (ReplaceSource-wrapped OriginalSources + SourceMapSources assembled by ConcatSource under CachedSource) with the V8 CPU profiler and sampling heap profiler. Three hot spots accounted for ~55% of all allocations; this addresses each without changing any output: - createMappingsSerializer built every mapping out of ~5 intermediate string concatenations and callers appended the result with `mappings += str` (31% of workload allocations). The full serializer now writes char codes into a growing Uint8Array via createMappingsWriter and materialises the mappings string once in finish(). The lines-only variant keeps the string fast paths (constant ";AACA" segments + cons append measured faster there) behind the same writer interface. The public string-returning createMappingsSerializer API is unchanged. - ReplaceSource.checkOriginalContent lazily split each source content into an array of line substrings (splitIntoLines, 11% of allocations). It now builds a Uint32Array of line-start offsets (exact-size, two-pass scan) and compares with String.startsWith — no per-line strings, no comparison slices. - ReplaceSource.streamChunks now honors finalSource like OriginalSource/RawSource: it emits position-only chunks (chunk === undefined) and returns the whole replaced source once, assembled in O(replacements) string operations by source(). This removes the per-token `code += chunk` cons chain in every enclosing consumer (ConcatSource, getSourceAndMap, streamAndGetSourceAndMap); all in-tree consumers already handle undefined chunks under finalSource. Measured on the profiling workload (interleaved A/B vs main, medians): total allocations 16.7 -> 6.6 MB/iter (-60%), wall time -14%. Isolated paths: ConcatSource.sourceAndMap -16..-20% time, ReplaceSource.sourceAndMap -8%, ReplaceSource.map(columns:true) -8%, OriginalSource.map allocations 37 -> 9 KB/op at equal time, columns:false paths at parity. All 89,888 tests pass; sources and maps are byte-identical to main across the fuzz suite. Claude-Session: https://claude.ai/code/session_01KDFBEvi3ctDSyHQ7f9izX5
🦋 Changeset detectedLatest commit: 77d2620 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #251 +/- ##
==========================================
+ Coverage 98.14% 98.29% +0.14%
==========================================
Files 25 25
Lines 2102 2283 +181
Branches 678 731 +53
==========================================
+ Hits 2063 2244 +181
Misses 37 37
Partials 2 2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will degrade performance by 23.35%
|
…sions
Addresses the three CI failures on the previous commit:
- lint: reformat the line-start scan loops for prettier 3.9.4 (the
version pinned by package-lock; a stale local install accepted the
old shape).
- codecov/project: the legacy lines-only serializer and several writer
branches lost integration coverage after the writer refactor. Add
unit tests asserting writer/serializer output equivalence over
branch-covering and buffer-growth event streams, plus exact-encoding
cases. createMappingsSerializer.js is back to 100% line coverage and
overall lib coverage now exceeds the main baseline.
- CodSpeed simulation regressions (benchmarks run under --no-opt, where
call indirection and extra passes cost far more than JIT-mode wall
time showed):
- createLinesOnlyMappingsWriter now inlines the encoding instead of
delegating to the string serializer per line
(original-source map({columns:false}) back to parity).
- checkOriginalContent builds line starts in a single pass with a
plain array (replace-source streamChunks back below base).
- ReplaceSource.streamChunks skips assembling the final source when
the caller passed source: false (getMap does); getSourceAndMap and
streamAndGetSourceAndMap explicitly override the flag because they
consume/cache the text — the streamChunks fallback forwards getMap
options into sourceAndMap, which the fuzz suite caught.
Interpreter-mode A/B vs main after these fixes (medians): replace-source
map() -12%, sourceAndMap() -32%, streamChunks(1000 replacements) -3%,
original-source map({columns:false}) parity, pipeline streamChunks
(columns off) -6%, full-serializer benchmark at parity (its flagged -27%
compares byte-identical code — environment skew, matching the report's
"different runtime environments" warning). Local ground truth for the
flagged webpack-20961 memory rows: allocations 160 MB -> 7.3 MB and
retained heap 42.4 MB -> 3.3 MB in favor of this branch (the writer
produces one flat mappings string instead of a huge retained cons tree).
All 89,895 tests pass; lint and types are green.
Claude-Session: https://claude.ai/code/session_01KDFBEvi3ctDSyHQ7f9izX5
2266774 to
77d2620
Compare
|
/easycla Generated by Claude Code |
|
Triage of the remaining CodSpeed warnings (all Memory mode; no Simulation/instruction-count regressions remain as of the latest runs): The flagged memory rows are measurement artifacts, not effects of this PR:
Happy to share the measurement scripts if useful. Given the above, the memory-mode warnings look acknowledgeable on the CodSpeed dashboard. Generated by Claude Code |
Summary
Deep-profiled a webpack-like chunk pipeline (
ReplaceSource-wrappedOriginalSources +SourceMapSources assembled byConcatSourceunderCachedSource) with the V8 CPU profiler and sampling heap profiler. Three sites accounted for ~55% of all allocations; this PR addresses each without changing any output:createMappingsSerializerbuilt every mapping out of ~5 intermediate string concatenations and all three internal callers appended the result withmappings += str(31% of workload allocations). The full serializer now writes char codes into a growingUint8Arrayvia a new internalcreateMappingsWriterand materialises the mappings string once infinish(). The lines-only variant keeps the string fast paths (constant";AACA"segments + cons append measured faster there) behind the same writer interface. The public string-returningcreateMappingsSerializerAPI is unchanged.ReplaceSource.checkOriginalContent. Instead of lazily splitting each source content into an array of line substrings (splitIntoLines, 11% of allocations), it builds an exact-sizeUint32Arrayof line-start offsets and compares withString.startsWith— no per-line strings, no comparison slices.ReplaceSource.streamChunkshonorsfinalSourcelikeOriginalSource/RawSourcealready do: it emits position-only chunks (chunk === undefined) and returns the whole replaced source once, assembled in O(replacements) string operations bysource(). This removes the per-tokencode += chunkcons chain in every enclosing consumer (ConcatSource,getSourceAndMap,streamAndGetSourceAndMap); all in-tree consumers already handleundefinedchunks underfinalSource.Measured (interleaved A/B vs
main, medians): workload total allocations 16.7 → 6.6 MB/iter (−60%), wall time −14%. Isolated paths:ConcatSource.sourceAndMap−16…−20% time (alloc 279 → 130 KB),ReplaceSource.sourceAndMap−8% (63 → 37 KB),ReplaceSource.map({columns:true})−8%,OriginalSource.map({columns:true})allocations 37 → 9 KB/op at equal time,columns:falsepaths at parity. Sources and maps are byte-identical tomainacross the fuzz suite.Rejected after measurement: a byte-writer for lines-only mappings (+50% CPU → hybrid instead); skipping
source()whenoptions.source === false(unsafe —CachedSourceforwards those options and caches the returned source). Known trade-off: standaloneReplaceSource.map({columns:false})is ~+12% (~0.03 ms) because returning the assembled source is required forCachedSourcecorrectness; webpack always consumesReplaceSourcethrough Concat/Cached wrappers, which all got faster.What kind of change does this PR introduce?
Performance improvement (patch, changeset included). No behavior change.
Did you add tests for your changes?
No new tests needed — the change is output-invariant and covered by the existing suites: all 89,888 tests pass (including the fuzz suite and 1373 snapshots), plus lint/types are green.
Does this PR introduce a breaking change?
No. The public
createMappingsSerializerAPI is unchanged;createMappingsWriteris a new internal export.ReplaceSource.streamChunksemittingundefinedchunks underfinalSource: truefollows the documented contract already used byOriginalSourceandRawSource.If relevant, what needs to be documented once your changes are merged or what have you already documented?
Nothing — internal performance change; a changeset describes it for the release notes.
Use of AI
This PR was written with Claude Code (profiling, implementation, and A/B measurement).
🤖 Generated with Claude Code
https://claude.ai/code/session_01KDFBEvi3ctDSyHQ7f9izX5
Generated by Claude Code