-
-
Notifications
You must be signed in to change notification settings - Fork 757
refactor(lazy-compilation): use POST request to transfer ids of active modules #12678
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(lazy-compilation): use POST request to transfer ids of active modules #12678
Conversation
✅ Deploy Preview for rspack canceled.
|
Rsdoctor Bundle Diff AnalysisFound 5 projects in monorepo, 0 projects with changes. 📊 Quick Summary
Generated by Rsdoctor GitHub Action |
📦 Binary Size-limit
🙈 Size remains the same at 47.90MB |
CodSpeed Performance ReportMerging this PR will not alter performanceComparing Summary
Footnotes
|
…module_id_goes_to_post_body
…module_id_goes_to_post_body
…module_id_goes_to_post_body
1. 如果其他中间件一个解析好了body,并挂载在req.body 上的花就直直接使用 req.body 2. packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts body 解析的时候不能简单的使用 string 的拼接,需要考虑多字节符号的截断的问题。
packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts 当req.body 非空时,直接假body中时一组 module ids,如果没有 body 自行拼接完,之后也 parse json string,返回 module ids 数组
7362078 to
5d8eeb5
Compare
chenjiahan
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR title: refactor -> refactor(lazy-compilation)
Co-authored-by: neverland <chenjiahan.jait@bytedance.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request refactors the lazy compilation mechanism to transmit active module IDs via POST request body instead of URL query parameters, addressing URL length limitations in browsers and servers. The change replaces EventSource-based communication with XMLHttpRequest using simple CORS requests (Content-Type: text/plain) to avoid preflight checks in cross-origin scenarios.
Changes:
- Migrated from EventSource to XMLHttpRequest with POST requests for activating lazy-compiled modules
- Implemented request queuing to batch module activations and avoid overwhelming the server
- Added backward compatibility by maintaining GET request support in the middleware
- Added comprehensive tests for cross-origin scenarios and handling many modules with long filenames
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/rspack/hot/lazy-compilation-web.js | Complete rewrite from EventSource to XMLHttpRequest with request queuing logic |
| packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts | Added POST body parsing with backward-compatible GET support |
| tests/e2e/cases/lazy-compilation/cross-origin/* | New test suite verifying cross-origin POST requests work correctly |
| tests/e2e/cases/lazy-compilation/active-lots-modules/* | New test for URL length limitation scenario with many long-named modules |
| tests/e2e/cases/lazy-compilation/custom-prefix/index.test.ts | Updated test to not check request method (now supports both GET and POST) |
| packages/rspack-test-tools/src/runner/web/index.ts | Added POST request proxying for lazy compilation in test environment |
| packages/rspack/etc/core.api.md | Marked lazyCompilationMiddleware as undocumented |
Comments suppressed due to low confidence (1)
packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts:231
- The middleware always sets Server-Sent Events (SSE) headers even for POST requests. For POST requests, the response should end immediately (which it does on line 247), but setting SSE headers like 'content-type: text/event-stream' is misleading and inappropriate for a simple POST response. These headers should only be set for GET requests that maintain a persistent connection.
req.socket.setNoDelay(true);
res.setHeader('content-type', 'text/event-stream');
res.writeHead(200);
res.write('\n');
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts
Outdated
Show resolved
Hide resolved
packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts
Outdated
Show resolved
Hide resolved
tests/e2e/cases/lazy-compilation/active-lots-modules/index.test.ts
Outdated
Show resolved
Hide resolved
packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts
Outdated
Show resolved
Hide resolved
packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts
Outdated
Show resolved
Hide resolved
…thub.com:web-infra-dev/rspack into refactor/lazy_active_module_id_goes_to_post_body
1d54eb6 to
bf0fc1f
Compare
…e modules (#12678) * refactor: use post transfer module ids * test: skip lazy active method * test: add lazy post request help * test: add large module id container * test: add case introduction * refactor: remove about controller * test: we are using post method now * refactor: rename * fix body parser (vibe-kanban 09797829) 1. 如果其他中间件一个解析好了body,并挂载在req.body 上的花就直直接使用 req.body 2. packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts body 解析的时候不能简单的使用 string 的拼接,需要考虑多字节符号的截断的问题。 * refactor read module ids from body (vibe-kanban a7874ac1) packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts 当req.body 非空时,直接假body中时一组 module ids,如果没有 body 自行拼接完,之后也 parse json string,返回 module ids 数组 * refactor: downgrading to lower web api * test: ✅ add lazy compilation active cors cases * chore: update test case doc * fix: we all need cors header no matter is simple request or not * refactor:delete cors header setting * refactor: set cors header should set by user * Update packages/rspack/hot/lazy-compilation-web.js Co-authored-by: neverland <chenjiahan.jait@bytedance.com> * test: fix case name * chore: api-extract update * fix: memory leak of requst listeners * refactor: remove event source handle logic * fix: jsdom XMLHTTPRequets need strict cors header * refactor: node lazy compilation client use post too --------- Co-authored-by: neverland <chenjiahan.jait@bytedance.com>
…e modules (#12678) * refactor: use post transfer module ids * test: skip lazy active method * test: add lazy post request help * test: add large module id container * test: add case introduction * refactor: remove about controller * test: we are using post method now * refactor: rename * fix body parser (vibe-kanban 09797829) 1. 如果其他中间件一个解析好了body,并挂载在req.body 上的花就直直接使用 req.body 2. packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts body 解析的时候不能简单的使用 string 的拼接,需要考虑多字节符号的截断的问题。 * refactor read module ids from body (vibe-kanban a7874ac1) packages/rspack/src/builtin-plugin/lazy-compilation/middleware.ts 当req.body 非空时,直接假body中时一组 module ids,如果没有 body 自行拼接完,之后也 parse json string,返回 module ids 数组 * refactor: downgrading to lower web api * test: ✅ add lazy compilation active cors cases * chore: update test case doc * fix: we all need cors header no matter is simple request or not * refactor:delete cors header setting * refactor: set cors header should set by user * Update packages/rspack/hot/lazy-compilation-web.js Co-authored-by: neverland <chenjiahan.jait@bytedance.com> * test: fix case name * chore: api-extract update * fix: memory leak of requst listeners * refactor: remove event source handle logic * fix: jsdom XMLHTTPRequets need strict cors header * refactor: node lazy compilation client use post too --------- Co-authored-by: neverland <chenjiahan.jait@bytedance.com>
Summary
The original method of passing module IDs to the active state via URL query strings may encounter limitations related to browser/server constraints or URL length. Therefore, we are refactoring this process to pass those IDs through the POST body instead.
To handle potential bursts of requests when activating modules, we will implement a request queue during the activation process. The steps are as follows:
compilingset.Related Links
Closes Issue #12649.
Checklist