mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-11-09 20:07:43 +08:00
Merge pull request #7552 from deepseek-harness/fix/webworker-live-import-bindings
fix(webworker): read named and default imports at each reference
This commit is contained in:
@@ -91,6 +91,8 @@ External packages installed for runtime use or embedded in distributed Host or b
|
||||
| [`diff`](https://github.com/kpdecker/jsdiff) | BSD-3-Clause |
|
||||
| [`dompurify`](https://github.com/cure53/DOMPurify) | (MPL-2.0 OR Apache-2.0) |
|
||||
| [`electron-updater`](https://github.com/electron-userland/electron-builder) | MIT |
|
||||
| [`eslint-scope`](https://github.com/eslint/js) | BSD-2-Clause |
|
||||
| [`eslint-visitor-keys`](https://github.com/eslint/js) | Apache-2.0 |
|
||||
| [`eventsource-parser`](https://github.com/rexxars/eventsource-parser) | MIT |
|
||||
| [`exceljs`](https://github.com/exceljs/exceljs) | MIT |
|
||||
| [`execa`](https://github.com/sindresorhus/execa) | MIT |
|
||||
@@ -220,6 +222,7 @@ External packages **directly declared** for development, tests, types, or toolin
|
||||
| [`@types/babel__code-frame`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/big.js`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/compression`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/estree`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/hosted-git-info`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/js-yaml`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/jsdom`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
|
||||
@@ -15,8 +15,8 @@
|
||||
en: 4e53164159b5ce41
|
||||
zh: 89582dc0294d0776
|
||||
/deepseek-ai-dsh-experimental-webworker-runtime/use-this-package:
|
||||
en: 24f1bb4adc4a9bb6
|
||||
zh: 4a2caa4bfe2d8911
|
||||
en: 9f4c76d6c4391037
|
||||
zh: c55611045f4667e2
|
||||
/deepseek-ai-dsh-experimental-webworker-runtime/model-experience:
|
||||
en: 61fce9b4c7dc96e6
|
||||
zh: dc2d931063a38327
|
||||
|
||||
@@ -29,7 +29,7 @@ Three build artifacts and one source-owned process layer:
|
||||
|
||||
- **`lib/index.js` (assembly library)** — `createWorkerHost`/`startWorkerHost` mount the base image and any ordered data overlays (`storage/`), register the image's lazy archives, install the module loader (`module-system/`) and the `process` shim, boot the tree through the image's own `dsh-app-boot`, and hand the tunnel its serving seams. Overlays may replace files only under `home/` and `workspace/`; they cannot replace the base manifest or modules. The image layout contract (`image-layout.ts`: virtual root, manifest path, lazy archive directory, empty directories) and the manifest (`image-manifest.ts`: profile, installation, the `lowered` wrapper-contract gate, lazy archives) are shared with the packer. The Worker boots the manifest's shipped profile as the launcher does: the image's app-boot initializes `home/profiles/<profile>/` from the shipped template, resolves bundles from the installation manifest the image carries alone, and composes the bundle layers, the profile patch, and the home patch. The profile context, the launch environment snapshot, and a `PluginPackages` service without a runtime resolution are provided as the launcher provides them; the Worker loader owns module resolution. The profile context carries no command-line overlay; the image's home patch (`home/cordis.patch.yml`), which the packer writes from `PLATFORM_ENTRY_CONFIG`, keeps JSONL Session logs plaintext. ConfigEditor persists overrides in the profile patch, reconciles each write, and Loader commits live values. The Worker commits `appReady` once the tunnel serves, so HMR loads and applies profile changes live; the Worker's ES module cache stays empty and the `process` shim reports no entry script, so HMR compares and replaces no loaded modules. Plugin management lists and switches the installation's optional bundles, and its package operations fail with the `execa` refusal. Configuration edits last for the current Worker lifetime.
|
||||
- **Lazy archives** — the manifest registers each page chunk the packer moved out of the base image by path and size, with its archive name relative to the image URL. Existence, stats, and listings answer at once; the first asynchronous read (`fs.promises.readFile`, `fs.promises.open`, `fs.createReadStream`, `fs.promises.cp`) fetches and inflates that archive once for every waiting reader, checks it against the registration, and mounts its files without publishing a mutation. A failed fetch rejects the read and leaves the files registered, so the next read fetches again. Until then, synchronous reads, writes, appends, truncation, chmod, rename, and link of a registered file fail with `ERR_DSH_VFS_DEFERRED` naming the archive; unlink and recursive removal drop the registration, and seeding the path replaces it. The archive name derives from its content digest, and the fetch checks only the registered paths and sizes, not the digest. An image that registers lazy archives must be passed as a URL.
|
||||
- **`lib/worker.js` (worker bundle)** — the assembly plus this package's Node-compatibility layer as one self-contained ES module. The module proxy table (`module-proxies.ts`) is the only platform fork: `node:*` builtins over VFS/tunnel/browser primitives, structural stubs that fail loud on the console for what a browser cannot do, and native/binary package replacements. `node:module` supplies `createRequire().resolve` and `.resolve.paths()` over the image package root, so unchanged packages can discover manifests without evaluating their modules. `node-addon-require-builtin` answers the Node v24.18.0 internal modules that the vendored Loader, app-boot, and HMR read without `--expose-internals`: the cascaded ESM loader's `resolveSync` and `import` over the Worker loader, with Node's `ERR_MODULE_NOT_FOUND` and `ERR_PACKAGE_PATH_NOT_EXPORTED` codes, and an empty `loadCache`; and the CommonJS loader, package reader, `modules` binding, and resolve-cache members that HMR replaces, which refuse when called. `process.versions.node` reports `24.18.0`, so `ModuleLoader.fromInternal()` and the Loader's `internal` reach that cascaded loader. `node:path.win32.isAbsolute` and `node:util.inherits` follow Node; other `win32` members throw. The `node:util` JSON-data comparator compares own enumerable properties, preserving Node admission decisions for recorded fields such as `__proto__`. The global `process` shim carries Node detection fields including `title`, preventing Worker execution from entering DOM-only branches; Node’s `global` alias references the same `globalThis` object for image-loaded packages. The pack-time parser reports statically named module requests, including module-scope direct calls of the form `createRequire(import.meta.url)('pkg')` through a named `node:module` or `module` import, to the packer's reachability walk. Stored, CommonJS-obtained, and rebased `createRequire` calls require image entry seeds. VFS mutations drive `node:fs` callback, polling, and promise watchers, whose listeners run in the async context where the watcher was created rather than that of the writing code; open descriptors retain file identity and access mode across rename, replacement, and unlink, `FileHandle.stat({ bigint: true })` reports the same device and inode identity as a path stat while the name still refers to that file, and `FileHandle.chmod()` updates the opened file identity; `readable-stream` supplies `node:stream/promises` completion and pipeline helpers and the stream state machine used by file streams and unchanged image packages such as Chokidar and readdirp. The maintained `string_decoder` package supplies `node:string_decoder`, retaining incomplete multibyte characters between chunks. AsyncLocalStorage carries sync-stack causality across `await` through the snapshot/restore faces the pack-time lowering injects. The worker holds no compiler: an image the packer did not lower is refused at mount. The callback `fs.realpath.native` alias resolves within the mounted VFS and supports `util.promisify`.
|
||||
- **`lib/worker.js` (worker bundle)** — the assembly plus this package's Node-compatibility layer as one self-contained ES module. The module proxy table (`module-proxies.ts`) is the only platform fork: `node:*` builtins over VFS/tunnel/browser primitives, structural stubs that fail loud on the console for what a browser cannot do, and native/binary package replacements. `node:module` supplies `createRequire().resolve` and `.resolve.paths()` over the image package root, so unchanged packages can discover manifests without evaluating their modules. `node-addon-require-builtin` answers the Node v24.18.0 internal modules that the vendored Loader, app-boot, and HMR read without `--expose-internals`: the cascaded ESM loader's `resolveSync` and `import` over the Worker loader, with Node's `ERR_MODULE_NOT_FOUND` and `ERR_PACKAGE_PATH_NOT_EXPORTED` codes, and an empty `loadCache`; and the CommonJS loader, package reader, `modules` binding, and resolve-cache members that HMR replaces, which refuse when called. `process.versions.node` reports `24.18.0`, so `ModuleLoader.fromInternal()` and the Loader's `internal` reach that cascaded loader. `node:path.win32.isAbsolute` and `node:util.inherits` follow Node; other `win32` members throw. The `node:util` JSON-data comparator compares own enumerable properties, preserving Node admission decisions for recorded fields such as `__proto__`. The global `process` shim carries Node detection fields including `title`, preventing Worker execution from entering DOM-only branches; Node’s `global` alias references the same `globalThis` object for image-loaded packages. The pack-time parser reports statically named module requests, including module-scope direct calls of the form `createRequire(import.meta.url)('pkg')` through a named `node:module` or `module` import, to the packer's reachability walk. Stored, CommonJS-obtained, and rebased `createRequire` calls require image entry seeds. Lowered bodies read named and default imports through the required module object at each reference, so a module in an import cycle observes a binding once the exporting module initializes it; in a module with such imports, an assignment to an imported binding, a direct `eval`, and a declaration using the reserved `__dsh$` prefix fail the pack. VFS mutations drive `node:fs` callback, polling, and promise watchers, whose listeners run in the async context where the watcher was created rather than that of the writing code; open descriptors retain file identity and access mode across rename, replacement, and unlink, `FileHandle.stat({ bigint: true })` reports the same device and inode identity as a path stat while the name still refers to that file, and `FileHandle.chmod()` updates the opened file identity; `readable-stream` supplies `node:stream/promises` completion and pipeline helpers and the stream state machine used by file streams and unchanged image packages such as Chokidar and readdirp. The maintained `string_decoder` package supplies `node:string_decoder`, retaining incomplete multibyte characters between chunks. AsyncLocalStorage carries sync-stack causality across `await` through the snapshot/restore faces the pack-time lowering injects. The worker holds no compiler: an image the packer did not lower is refused at mount. The callback `fs.realpath.native` alias resolves within the mounted VFS and supports `util.promisify`.
|
||||
- **`src/shell/` (the worker's own process layer)** — a browser worker cannot fork, so `node:child_process` is not a stub but an implementation: `spawn` starts the command in its own Web Worker — this same bundle, told by its first frame to be a shell process — and reports it through the `ChildProcess` surface the subprocess service consumes. The command runs off the host's thread, `SIGKILL` terminates it whatever it is doing, and it reaches the VFS only by message (the host serves those frames). Worker platform executables preserve native-package protocols such as Landlock without replacing their JavaScript packages or coupling their implementations to `node:child_process`; `xdg-open` sends one VFS text file to the page's read-only viewer, so desktop-opener actions such as **Open configuration file** show the file in the page; ordinary commands use the package's evaluator and coreutils command table. The grammar is `@yarnpkg/parsers`' `parseShell`, while `execSync`/`fork` still refuse because they need a real process.
|
||||
- **`lib/client.js` (page half)** — startup has two independent stages. `chooseWorkerHostSource({ image?, fixtureManifest? })` optionally owns the boot barrier and fixture manifest: without `preview-fixture` it waits at the source chooser, while a valid query selects directly; either path returns ordered overlays. `connectWorkerHost(worker, { image?, overlays? })` remains the public base-runtime connector; callers that skip the chooser get an empty overlay list. `apps/web` invokes both and supplies its statically bundled Worker. The opening `init` frame carries the base and ordered overlay URLs, the boot payload delivers the structured index-injection table, and `applyIndexInjections` executes it before the shell entry runs. Script preload rows are advisory and skipped because `/plugins` resources resolve only through the tunnel; `loadBundle` fetches each combo on first demand, embeds its tunnel-only source map as a Base64 data URL, and executes the script as a Blob. Before the shell entry runs, the page reads `GET /__assets__`, which the tunnel answers with an uncompressed tar of every package asset in the image, and supplies `resolveAsset`, which maps each `plugins/<package>/assets/<path>` reference a bundle passes to `require.asset` to a Blob URL of those bytes. The worker's unsolicited `view-text` frame opens the page's modal text viewer. The tunnel also exposes fetch-shaped transport, the independent file-upload carrier, and the API client. Request frames preserve Blob bodies through structured clone and transfer `ReadableStream<Uint8Array>` ownership. The Host Worker streams both forms into the route, so neither browser thread creates a complete byte array for a generic-file upload. A Remote stream's uplink crosses as `stream-uplink-item` / `stream-uplink-end` frames into an unbounded `TunnelUplink` that drops items arriving after `end`: the page and its worker are one trusted same-origin boundary, so the Host inbox cap does not apply.
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ kind: "package-library"
|
||||
|
||||
- **`lib/index.js`(装配库)**——`createWorkerHost`/`startWorkerHost` 挂载基础镜像和按序排列的数据 overlays(`storage/`)、登记镜像的懒加载归档、安装模块加载器(`module-system/`)与 `process` shim、经镜像自带的 `dsh-app-boot` 启动插件树,并把服务 seam 交给隧道。Overlay 只能替换 `home/` 与 `workspace/` 下的文件,不能替换基础 manifest 或模块。镜像布局契约(`image-layout.ts`:虚拟根、manifest 路径、懒加载归档目录、空目录)与 manifest(`image-manifest.ts`:profile、安装清单、`lowered` 包装契约门、懒加载归档)与 packer 共享。Worker 按启动器的方式启动 manifest 指定的随附 profile:镜像里的 app-boot 用随附模板初始化 `home/profiles/<profile>/`,从镜像单独携带的安装 manifest 解析 bundle,并合成各 bundle 层、profile patch 与 home patch。profile 上下文、启动环境快照,以及不带运行时 resolution 的 `PluginPackages` 服务都按启动器的方式提供;模块解析由 Worker 加载器负责。profile 上下文不带命令行 overlay;packer 按 `PLATFORM_ENTRY_CONFIG` 写入镜像的 home patch(`home/cordis.patch.yml`),使 JSONL Session 日志保持明文。ConfigEditor 将覆盖值持久化到 profile patch,协调每次写入,Loader 提交实时值。Worker 在隧道开始服务后提交 `appReady`,因此 HMR 可以加载 profile 变更并实时应用;Worker 的 ES module 缓存始终为空,`process` shim 也不报告入口脚本,因此 HMR 不比较、也不替换已加载的模块。插件管理可以列出并开关安装提供的 optional bundle,其包操作会因 `execa` 拒绝而失败。配置修改在当前 Worker 生命周期内有效。
|
||||
- **懒加载归档**——manifest 按路径和大小登记 packer 移出基础镜像的每个页面 chunk,并给出相对镜像 URL 的归档名。存在性、stat 和目录列表立即作答;第一次异步读取(`fs.promises.readFile`、`fs.promises.open`、`fs.createReadStream`、`fs.promises.cp`)为所有等待中的读取方只拉取并解压一次该归档,按登记内容校验后挂载其文件,不发布 mutation。拉取失败时本次读取被拒绝,文件保持登记状态,下次读取会重新拉取。在此之前,对已登记文件的同步读取、写入、追加、截断、chmod、rename 和 link 会以 `ERR_DSH_VFS_DEFERRED` 失败,错误中写明归档名;unlink 和递归删除会移除登记,再次 seed 该路径会替换登记。归档名由内容摘要派生,拉取时只校验登记的路径和大小,不校验摘要。登记了懒加载归档的镜像必须以 URL 传入。
|
||||
- **`lib/worker.js`(worker 束)**——装配库加本包的 Node 兼容层,合成一个自含 ES module。模块代理表(`module-proxies.ts`)是唯一平台叉口:`node:*` 内建走 VFS、隧道和浏览器原语,浏览器做不到的走结构化 stub(调用即在 console 报错并抛出),native/binary 包则替换执行后端。`node:module` 在镜像 package 根之上提供 `createRequire().resolve` 与 `.resolve.paths()`,使未修改的包无需执行目标模块即可发现 manifest。`node-addon-require-builtin` 应答 vendor Loader、app-boot 和 HMR 在没有 `--expose-internals` 时读取的 Node v24.18.0 internal 模块:cascaded ESM loader 的 `resolveSync` 和 `import` 经 Worker 加载器完成,失败时带 Node 的 `ERR_MODULE_NOT_FOUND` 与 `ERR_PACKAGE_PATH_NOT_EXPORTED` 错误码,`loadCache` 为空;HMR 会替换的 CommonJS loader、package reader、`modules` binding 与 resolve cache 成员被调用时拒绝。`process.versions.node` 报告 `24.18.0`,因此 `ModuleLoader.fromInternal()` 和 Loader 的 `internal` 拿到的是这个 cascaded loader。`node:path.win32.isAbsolute` 与 `node:util.inherits` 与 Node 一致,其余 `win32` 成员抛出。`node:util` 的 JSON 数据比较器检查可枚举的自有属性,使 `__proto__` 等记录字段的准入判定与 Node 一致。全局 `process` shim 带有包括 `title` 在内的 Node 环境识别字段,避免 Worker 执行误入仅适用于 DOM 的分支;镜像中加载的包通过 Node 的 `global` 别名访问同一个 `globalThis` 对象。pack 期解析器会把名称静态可知的模块请求报告给 packer 的可达性遍历,其中包括通过 `node:module` 或 `module` 具名导入在模块作用域直接发起的 `createRequire(import.meta.url)('pkg')` 调用。保存、经 CommonJS 获取或另设基准的 `createRequire` 调用需要镜像入口种子。VFS mutation 驱动 `node:fs` 的 callback、polling 和 promise watcher,监听回调运行在创建 watcher 时的异步上下文中,而不是写入方代码的上下文中;打开的 descriptor 在 rename、replacement 和 unlink 后仍保留文件身份与访问模式,只要文件名仍指向该文件,`FileHandle.stat({ bigint: true })` 报告的 device 与 inode 身份就与路径 stat 相同,`FileHandle.chmod()` 则更新打开文件身份的权限;`readable-stream` 提供 `node:stream/promises` 的完成等待与管道方法,以及文件流以及 Chokidar、readdirp 等未修改镜像包所用的流状态机。维护中的 `string_decoder` 包提供 `node:string_decoder`,在数据块之间保留不完整的多字节字符。AsyncLocalStorage 经 pack 时降低注入的 snapshot/restore 面在 `await` 间携带同步栈因果。worker 不带编译器:packer 未降低的镜像在挂载时被拒。回调式 `fs.realpath.native` 别名在已挂载的 VFS 内解析路径,并支持 `util.promisify`。
|
||||
- **`lib/worker.js`(worker 束)**——装配库加本包的 Node 兼容层,合成一个自含 ES module。模块代理表(`module-proxies.ts`)是唯一平台叉口:`node:*` 内建走 VFS、隧道和浏览器原语,浏览器做不到的走结构化 stub(调用即在 console 报错并抛出),native/binary 包则替换执行后端。`node:module` 在镜像 package 根之上提供 `createRequire().resolve` 与 `.resolve.paths()`,使未修改的包无需执行目标模块即可发现 manifest。`node-addon-require-builtin` 应答 vendor Loader、app-boot 和 HMR 在没有 `--expose-internals` 时读取的 Node v24.18.0 internal 模块:cascaded ESM loader 的 `resolveSync` 和 `import` 经 Worker 加载器完成,失败时带 Node 的 `ERR_MODULE_NOT_FOUND` 与 `ERR_PACKAGE_PATH_NOT_EXPORTED` 错误码,`loadCache` 为空;HMR 会替换的 CommonJS loader、package reader、`modules` binding 与 resolve cache 成员被调用时拒绝。`process.versions.node` 报告 `24.18.0`,因此 `ModuleLoader.fromInternal()` 和 Loader 的 `internal` 拿到的是这个 cascaded loader。`node:path.win32.isAbsolute` 与 `node:util.inherits` 与 Node 一致,其余 `win32` 成员抛出。`node:util` 的 JSON 数据比较器检查可枚举的自有属性,使 `__proto__` 等记录字段的准入判定与 Node 一致。全局 `process` shim 带有包括 `title` 在内的 Node 环境识别字段,避免 Worker 执行误入仅适用于 DOM 的分支;镜像中加载的包通过 Node 的 `global` 别名访问同一个 `globalThis` 对象。pack 期解析器会把名称静态可知的模块请求报告给 packer 的可达性遍历,其中包括通过 `node:module` 或 `module` 具名导入在模块作用域直接发起的 `createRequire(import.meta.url)('pkg')` 调用。保存、经 CommonJS 获取或另设基准的 `createRequire` 调用需要镜像入口种子。降低后的模块体在每次引用具名导入与 default 导入时,都经所 require 的模块对象读取,因此循环导入中的模块能在导出方完成初始化后读到该绑定;在含此类导入的模块中,对导入绑定赋值、直接调用 `eval`、声明使用保留前缀 `__dsh$` 的名字,都会使 pack 失败。VFS mutation 驱动 `node:fs` 的 callback、polling 和 promise watcher,监听回调运行在创建 watcher 时的异步上下文中,而不是写入方代码的上下文中;打开的 descriptor 在 rename、replacement 和 unlink 后仍保留文件身份与访问模式,只要文件名仍指向该文件,`FileHandle.stat({ bigint: true })` 报告的 device 与 inode 身份就与路径 stat 相同,`FileHandle.chmod()` 则更新打开文件身份的权限;`readable-stream` 提供 `node:stream/promises` 的完成等待与管道方法,以及文件流以及 Chokidar、readdirp 等未修改镜像包所用的流状态机。维护中的 `string_decoder` 包提供 `node:string_decoder`,在数据块之间保留不完整的多字节字符。AsyncLocalStorage 经 pack 时降低注入的 snapshot/restore 面在 `await` 间携带同步栈因果。worker 不带编译器:packer 未降低的镜像在挂载时被拒。回调式 `fs.realpath.native` 别名在已挂载的 VFS 内解析路径,并支持 `util.promisify`。
|
||||
- **`src/shell/`(worker 自己的进程层)**——浏览器 worker 无法 fork,所以 `node:child_process` 不是 stub 而是实现:`spawn` 把命令放进它自己的 Web Worker——就是这同一个束,由首帧告诉它「你是 shell 进程」——并以 subprocess 服务消费的 `ChildProcess` 面报告结果。命令不占宿主线程,`SIGKILL` 不管它在干什么都能终止它,而它只能靠消息触达 VFS(由宿主应答这些帧)。Worker 平台 executable 在不替换 JavaScript 包、也不把具体实现耦合进 `node:child_process` 的情况下保持 Landlock 等 native 包协议;`xdg-open` 把一个 VFS 文本文件交给页面的只读查看器,因此 **打开配置文件** 等桌面打开操作会在页面内显示该文件;普通命令使用本包的求值器与 coreutils 命令表。语法来自 `@yarnpkg/parsers` 的 `parseShell`,而 `execSync`/`fork` 依然拒绝,因为它们需要真进程。
|
||||
- **`lib/client.js`(页面半)**——启动分为相互独立的两段。`chooseWorkerHostSource({ image?, fixtureManifest? })` 可选地拥有 boot barrier 与 fixture manifest:没有 `preview-fixture` 时停在来源选择面板,合法 query 则直接选择;两条路径都返回按序排列的 overlays。`connectWorkerHost(worker, { image?, overlays? })` 仍是公开的基础运行态连接器;调用方跳过选择器时 overlay 列表为空。`apps/web` 调用这两段并提供静态打包的 Worker。开局 `init` 帧携带基础镜像与按序排列的 overlay URL,boot 载荷送达结构化 index 注入表,`applyIndexInjections` 在壳入口运行前逐行执行。脚本 preload 行只是提示,因此会被跳过:`/plugins` 资源只能经 tunnel 解析,`loadBundle` 会在首次需要时获取 combo、把仅 tunnel 可达的 sourcemap 内嵌为 Base64 data URL,再以 Blob 执行脚本。在 shell 入口运行之前,页面读取 `GET /__assets__`(隧道以镜像中所有包资源组成的未压缩 tar 回答),并提供 `resolveAsset`,把 bundle 传给 `require.asset` 的每个 `plugins/<package>/assets/<path>` 引用映射为这些字节的 Blob URL。worker 主动发出的 `view-text` 帧会打开页面的模态文本查看器。Tunnel 还暴露 fetch 形式的传输、独立文件上传载体和 API 客户端。请求帧通过结构化克隆保留 Blob 请求体,并转移 `ReadableStream<Uint8Array>` 的所有权。Host Worker 将两种请求体都逐块送入路由,因此通用文件上传不会在任何浏览器线程创建完整字节数组。Remote 流的上行以 `stream-uplink-item` / `stream-uplink-end` 帧进入没有上限的 `TunnelUplink`,`end` 之后到达的项被丢弃:页面与其 worker 是同一个可信的同源边界,Host 的 inbox 上限在此不适用。
|
||||
|
||||
|
||||
@@ -33,6 +33,8 @@
|
||||
"@yarnpkg/parsers": "^3.1.0",
|
||||
"acorn": "^8.18.0",
|
||||
"buffer": "^6.0.3",
|
||||
"eslint-scope": "^9.1.2",
|
||||
"eslint-visitor-keys": "^5.0.1",
|
||||
"picomatch": "^4.0.7",
|
||||
"readable-stream": "^4.7.0",
|
||||
"@deepseek-ai/dsh-client-web": "workspace:*",
|
||||
@@ -63,6 +65,7 @@
|
||||
"@deepseek-ai/dsh-subagent": "workspace:*",
|
||||
"@deepseek-ai/dsh-subprocess-local": "workspace:*",
|
||||
"@deepseek-ai/node-addon-system": "workspace:~",
|
||||
"@types/estree": "^1.0.9",
|
||||
"@types/picomatch": "^4.0.3",
|
||||
"@types/readable-stream": "^4.0.24",
|
||||
"chokidar": "5.0.0"
|
||||
|
||||
@@ -10,12 +10,23 @@
|
||||
* so **line numbers survive**: a stack frame in a transformed module points at
|
||||
* the same line as the built artifact it came from.
|
||||
*
|
||||
* Named and default imports stay live: every reference reads the required
|
||||
* module object when it runs (`__dsh$m1["x"]`), never a copy taken at the
|
||||
* import statement, so a module in an import cycle sees a binding once the
|
||||
* exporting module has initialized it, as ES modules do. Scope analysis tells
|
||||
* those references apart from same-named locals that shadow them; it runs only
|
||||
* for a module that declares or assigns an import's name somewhere, because
|
||||
* elsewhere every read of the name is the import.
|
||||
*
|
||||
* The image packer is this transform's only caller: it lowers every JavaScript
|
||||
* entry it packs and records `LOWERING_VERSION` in the image manifest, so the
|
||||
* worker wraps those bodies without carrying a compiler of its own.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/compile/transform
|
||||
*/
|
||||
import { parse } from 'acorn'
|
||||
import { parse, type Identifier, type Literal, type Program } from 'acorn'
|
||||
import { analyze } from 'eslint-scope'
|
||||
import { KEYS } from 'eslint-visitor-keys'
|
||||
import type * as ESTree from 'estree'
|
||||
|
||||
const HELPER_SOURCE: Record<string, string> = {
|
||||
def: 'const __dsh$def=(t,k,get)=>Object.defineProperty(t,k,{enumerable:true,configurable:true,get});',
|
||||
@@ -60,6 +71,28 @@ interface Binding {
|
||||
readonly local: string
|
||||
}
|
||||
|
||||
/** A named or default import, read through its module object at every reference. */
|
||||
interface ImportedBinding {
|
||||
/** Temporary holding the required module object. */
|
||||
readonly held: string
|
||||
readonly kind: 'named' | 'default'
|
||||
/** Export name a named import reads. */
|
||||
readonly imported: string
|
||||
}
|
||||
|
||||
/** An identifier in a read position that spells an import's name. */
|
||||
interface ImportCandidate {
|
||||
readonly node: Node
|
||||
readonly parent: Node
|
||||
readonly key: string | undefined
|
||||
/** Start of the enclosing expression statement when that statement sits in a statement list. */
|
||||
readonly statementStart: number | undefined
|
||||
readonly binding: ImportedBinding
|
||||
}
|
||||
|
||||
/** Prefix of the identifiers this transform emits; a module declaring one would capture them. */
|
||||
const RESERVED_PREFIX = '__dsh$'
|
||||
|
||||
class Transformer {
|
||||
private readonly edits: Edit[] = []
|
||||
private readonly source: string
|
||||
@@ -71,6 +104,18 @@ class Transformer {
|
||||
private readonly moduleRequests = new Set<string>()
|
||||
private readonly metaResolveRequests = new Set<string>()
|
||||
private readonly createRequireBindings = new Set<string>()
|
||||
/** Module temporary per import declaration with specifiers, assigned before traversal. */
|
||||
private readonly importModules = new Map<object, string>()
|
||||
/** Named and default imports by local name. */
|
||||
private readonly importedBindings = new Map<string, ImportedBinding>()
|
||||
/** Read-position identifiers spelling an import's name, rewritten after traversal. */
|
||||
private readonly importCandidates: ImportCandidate[] = []
|
||||
/**
|
||||
* Set when an import name (or the reserved prefix) is declared or assigned.
|
||||
* Only then can a candidate be a shadowing local, so only then does scope
|
||||
* analysis decide which candidates read the import.
|
||||
*/
|
||||
private scopeAnalysisNeeded = false
|
||||
|
||||
constructor(source: string, private readonly path: string) {
|
||||
// A `#!` line is only legal at offset zero, and the prologue takes that spot;
|
||||
@@ -157,30 +202,85 @@ class Transformer {
|
||||
}
|
||||
const source = node.source as Node
|
||||
const request = `require(${this.literal(source)})`
|
||||
const specifiers = node.specifiers as Node[]
|
||||
if (specifiers.length === 0) {
|
||||
const held = this.importModules.get(node)
|
||||
if (held === undefined) {
|
||||
this.replace(node.start, node.end, `${request};`)
|
||||
return
|
||||
}
|
||||
const held = this.moduleTemp()
|
||||
// Named and default imports declare nothing here: each reference reads the
|
||||
// held module object instead (`importReference`).
|
||||
const lines = [`const ${held}=${request};`]
|
||||
for (const specifier of specifiers) {
|
||||
const local = (specifier.local as Node).name as string
|
||||
if (specifier.type === 'ImportDefaultSpecifier') {
|
||||
lines.push(`const ${local}=${this.helper('default')}(${held});`)
|
||||
continue
|
||||
}
|
||||
for (const specifier of node.specifiers as Node[]) {
|
||||
if (specifier.type === 'ImportNamespaceSpecifier') {
|
||||
lines.push(`const ${local}=${this.helper('ns')}(${held});`)
|
||||
continue
|
||||
lines.push(`const ${nameOf(specifier.local as Node)}=${this.helper('ns')}(${held});`)
|
||||
}
|
||||
const imported = specifier.imported as Node
|
||||
const name = imported.type === 'Identifier' ? imported.name as string : imported.value as string
|
||||
lines.push(`const ${local}=${held}[${JSON.stringify(name)}];`)
|
||||
}
|
||||
this.replace(node.start, node.end, lines.join(''))
|
||||
}
|
||||
|
||||
/**
|
||||
* The expression that reads an import when the reference runs. The default
|
||||
* form is parenthesized so it stays one operand in any position, including
|
||||
* `new` callees and member objects.
|
||||
*/
|
||||
private importAccess(binding: ImportedBinding): string {
|
||||
return binding.kind === 'default'
|
||||
? `(${this.helper('default')}(${binding.held}))`
|
||||
: `${binding.held}[${JSON.stringify(binding.imported)}]`
|
||||
}
|
||||
|
||||
/** @returns How a local export reads its binding: imports through their module object. */
|
||||
private bindingAccess(local: string): string {
|
||||
const imported = this.importedBindings.get(local)
|
||||
return imported === undefined ? local : this.importAccess(imported)
|
||||
}
|
||||
|
||||
/**
|
||||
* Record an identifier that spells an import's name in a read position, and
|
||||
* flag one in a declaration or assignment position (or one using the
|
||||
* reserved prefix there), which makes scope analysis decide the candidates.
|
||||
* @param node - The identifier.
|
||||
* @param parent - Its parent node.
|
||||
* @param key - The parent property holding it.
|
||||
* @param statementStart - Start of the enclosing expression statement when that statement sits in a statement list.
|
||||
*/
|
||||
private identifier(node: Node, parent: Node | undefined, key: string | undefined, statementStart: number | undefined): void {
|
||||
if (parent === undefined || this.importedBindings.size === 0) return
|
||||
const name = node.name as string
|
||||
const binding = this.importedBindings.get(name)
|
||||
if (binding === undefined && !name.startsWith(RESERVED_PREFIX)) return
|
||||
if (isBindingPosition(parent, key)) {
|
||||
this.scopeAnalysisNeeded = true
|
||||
return
|
||||
}
|
||||
if (binding !== undefined && isReadPosition(parent, key)) {
|
||||
this.importCandidates.push({ node, parent, key, statementStart, binding })
|
||||
}
|
||||
}
|
||||
|
||||
/** Flag a declared or assigned name that could shadow an import or capture an emitted temporary. */
|
||||
private bindingOccurrence(name: string): void {
|
||||
if (this.importedBindings.size === 0) return
|
||||
if (this.importedBindings.has(name) || name.startsWith(RESERVED_PREFIX)) this.scopeAnalysisNeeded = true
|
||||
}
|
||||
|
||||
/** Rewrite one identifier that reads an import. */
|
||||
private importReference({ node, parent, key, statementStart, binding }: ImportCandidate): void {
|
||||
const access = this.importAccess(binding)
|
||||
// A shorthand property holds the name as both key and value; only the value
|
||||
// is a candidate.
|
||||
if (parent.type === 'Property' && parent.shorthand === true) {
|
||||
this.replace(node.start, node.end, `${nameOf(node)}:${access}`)
|
||||
return
|
||||
}
|
||||
const called = (parent.type === 'CallExpression' && key === 'callee')
|
||||
|| (parent.type === 'TaggedTemplateExpression' && key === 'tag')
|
||||
// `(0, m.f)()` calls with `this` undefined, as calling an imported binding does.
|
||||
const text = called && binding.kind === 'named' ? `(0,${access})` : access
|
||||
// A leading parenthesis would extend a preceding statement that ends without a semicolon.
|
||||
this.replace(node.start, node.end, text.startsWith('(') && node.start === statementStart ? `;${text}` : text)
|
||||
}
|
||||
|
||||
private exportNamed(node: Node): void {
|
||||
this.moduleSyntax = true
|
||||
const declaration = node.declaration as Node | null
|
||||
@@ -320,16 +420,27 @@ class Transformer {
|
||||
functionDepth: number
|
||||
moduleScope: boolean
|
||||
statement?: Node
|
||||
}): void {
|
||||
/** Start of the enclosing expression statement when it sits in a statement list. */
|
||||
statementStart?: number | undefined
|
||||
}, parent?: Node, key?: string): void {
|
||||
if (node === null || typeof node !== 'object') return
|
||||
if (Array.isArray(node)) {
|
||||
for (const child of node) this.visit(child, context)
|
||||
for (const child of node) this.visit(child, context, parent, key)
|
||||
return
|
||||
}
|
||||
const record = node as Node
|
||||
if (typeof record.type !== 'string') return
|
||||
let next = context
|
||||
switch (record.type) {
|
||||
case 'Identifier': this.identifier(record, parent, key, context.statementStart); break
|
||||
case 'ObjectPattern':
|
||||
// A property value here is a declaration or assignment target, which its
|
||||
// own parent (a Property) cannot tell apart from an object literal value.
|
||||
for (const property of record.properties as Node[]) {
|
||||
const value = property.value as Node | undefined
|
||||
if (value?.type === 'Identifier') this.bindingOccurrence(value.name as string)
|
||||
}
|
||||
break
|
||||
case 'ImportDeclaration': this.importDeclaration(record); break
|
||||
case 'ExportNamedDeclaration': this.exportNamed(record); break
|
||||
case 'ExportDefaultDeclaration': this.exportDefault(record); break
|
||||
@@ -350,6 +461,13 @@ class Transformer {
|
||||
// only a direct module-scope createRequire call with the importer URL.
|
||||
const callee = record.callee as Node
|
||||
const callArguments = record.arguments as Node[]
|
||||
// Module code is strict, so `eval` cannot be rebound: a call through the
|
||||
// bare name is a direct eval, whose code could name an import that no
|
||||
// longer exists as a local. An optional call (`eval?.()`) is indirect.
|
||||
if (callee.type === 'Identifier' && callee.name === 'eval' && record.optional !== true
|
||||
&& this.importedBindings.size > 0) {
|
||||
this.fail('direct eval cannot see imports once they are read through their module', record.start)
|
||||
}
|
||||
if (this.isRequireCall(callee, context.moduleScope) && callArguments.length === 1
|
||||
&& typeof callArguments[0]?.value === 'string') {
|
||||
this.moduleRequests.add(callArguments[0].value)
|
||||
@@ -421,10 +539,12 @@ class Transformer {
|
||||
break
|
||||
default: break
|
||||
}
|
||||
if (record.type === 'ExpressionStatement') next = { ...next, statement: record }
|
||||
for (const [key, value] of Object.entries(record)) {
|
||||
if (key === 'type' || key === 'start' || key === 'end') continue
|
||||
this.visit(value, next)
|
||||
if (record.type === 'ExpressionStatement') {
|
||||
next = { ...next, statement: record, statementStart: inStatementList(parent, key) ? record.start : undefined }
|
||||
}
|
||||
for (const [childKey, value] of Object.entries(record)) {
|
||||
if (childKey === 'type' || childKey === 'start' || childKey === 'end') continue
|
||||
this.visit(value, next, record, childKey)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -450,18 +570,82 @@ class Transformer {
|
||||
|| (moduleScope && this.isCreateRequireCall(callee))
|
||||
}
|
||||
|
||||
private indexCreateRequireImports(program: Node): void {
|
||||
for (const statement of program.body as Node[]) {
|
||||
private indexCreateRequireImports(program: Program): void {
|
||||
for (const statement of program.body) {
|
||||
if (statement.type !== 'ImportDeclaration') continue
|
||||
const source = statement.source as Node
|
||||
const source = statement.source
|
||||
if (source.value !== 'node:module' && source.value !== 'module') continue
|
||||
for (const specifier of statement.specifiers as Node[]) {
|
||||
if (specifier.type !== 'ImportSpecifier' || nameOf(specifier.imported as Node) !== 'createRequire') continue
|
||||
this.createRequireBindings.add(nameOf(specifier.local as Node))
|
||||
for (const specifier of statement.specifiers) {
|
||||
if (specifier.type !== 'ImportSpecifier' || specifierName(specifier.imported) !== 'createRequire') continue
|
||||
this.createRequireBindings.add(specifier.local.name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Give every import declaration its module temporary and record its named and
|
||||
* default bindings before traversal, because references may precede the
|
||||
* declaration in source order.
|
||||
*/
|
||||
private indexImports(program: Program): void {
|
||||
for (const statement of program.body) {
|
||||
if (statement.type !== 'ImportDeclaration' || statement.specifiers.length === 0) continue
|
||||
const held = this.moduleTemp()
|
||||
this.importModules.set(statement, held)
|
||||
for (const specifier of statement.specifiers) {
|
||||
if (specifier.type === 'ImportDefaultSpecifier') {
|
||||
this.importedBindings.set(specifier.local.name, { held, kind: 'default', imported: 'default' })
|
||||
} else if (specifier.type === 'ImportSpecifier') {
|
||||
this.importedBindings.set(specifier.local.name, { held, kind: 'named', imported: specifierName(specifier.imported) })
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide with scope analysis which candidates read an import.
|
||||
*
|
||||
* Refuses an assignment to an import, which live reads cannot express, and a
|
||||
* declaration using the reserved prefix, which would capture an emitted
|
||||
* module temporary. Direct `eval` is refused during traversal.
|
||||
* @param program - The parsed module, which gains node ranges here.
|
||||
* @returns The identifier nodes that read a named or default import.
|
||||
*/
|
||||
private resolveImportReads(program: Program): ReadonlySet<object> {
|
||||
assignRanges(program)
|
||||
// acorn's node types model directives differently from `@types/estree`; the
|
||||
// tree itself is ESTree. eslint-scope distinguishes only ES5 and ES2015+
|
||||
// scoping, so any later edition selects the module rules.
|
||||
const manager = analyze(program as ESTree.Program, { ecmaVersion: 2022, sourceType: 'module', childVisitorKeys: KEYS })
|
||||
for (const scope of manager.scopes) {
|
||||
for (const variable of scope.variables) {
|
||||
const definition = variable.defs[0]
|
||||
if (definition === undefined || !variable.name.startsWith(RESERVED_PREFIX)) continue
|
||||
this.fail(`${variable.name} uses the transform's reserved ${RESERVED_PREFIX} prefix`, this.rangeStart(definition.name))
|
||||
}
|
||||
}
|
||||
const reads = new Set<object>()
|
||||
const moduleScope = manager.scopes.find(scope => scope.type === 'module')
|
||||
for (const variable of moduleScope?.variables ?? []) {
|
||||
if (!this.importedBindings.has(variable.name) || variable.defs[0]?.type !== 'ImportBinding') continue
|
||||
for (const reference of variable.references) {
|
||||
if (reference.isWrite()) this.fail(`the import ${variable.name} cannot be assigned`, this.rangeStart(reference.identifier))
|
||||
reads.add(reference.identifier)
|
||||
}
|
||||
}
|
||||
return reads
|
||||
}
|
||||
|
||||
/**
|
||||
* @param node - A node scope analysis reports.
|
||||
* @returns Its source offset; `assignRanges` gave every node a range.
|
||||
*/
|
||||
private rangeStart(node: ESTree.BaseNode): number {
|
||||
const start = node.range?.[0]
|
||||
if (start === undefined) this.fail('scope analysis reported a node without a source range', 0)
|
||||
return start
|
||||
}
|
||||
|
||||
run(): string {
|
||||
// Transforming a lowered body again would nest the protocol inside itself:
|
||||
// it still runs, only slower and unreadable, so a mis-wired manifest must
|
||||
@@ -469,29 +653,38 @@ class Transformer {
|
||||
if (this.source.includes(`${ALS}.pause(`) || this.source.includes('__als$')) {
|
||||
this.fail('the module is already lowered; check the image manifest wiring', 0)
|
||||
}
|
||||
let program: Node
|
||||
let program: Program
|
||||
try {
|
||||
program = parse(this.source, {
|
||||
ecmaVersion: 'latest',
|
||||
sourceType: 'module',
|
||||
allowAwaitOutsideFunction: true,
|
||||
}) as unknown as Node
|
||||
})
|
||||
} catch (reason) {
|
||||
this.fail(`parse failed: ${(reason as Error).message}`, 0)
|
||||
}
|
||||
this.indexCreateRequireImports(program)
|
||||
this.indexImports(program)
|
||||
this.visit(program, { asyncGenerator: false, functionDepth: 0, moduleScope: true })
|
||||
// Without a declaration or assignment of an import's name, nothing can
|
||||
// shadow it, so every candidate reads the import.
|
||||
const reads = this.scopeAnalysisNeeded ? this.resolveImportReads(program) : undefined
|
||||
for (const candidate of this.importCandidates) {
|
||||
if (reads === undefined || reads.has(candidate.node)) this.importReference(candidate)
|
||||
}
|
||||
if (this.edits.length === 0 && !this.moduleSyntax) return this.source
|
||||
|
||||
const prologue: string[] = []
|
||||
if (this.moduleSyntax) prologue.push('"use strict";Object.defineProperty(exports,"__esModule",{value:true});')
|
||||
// Reading an exported import can register the default helper, so the
|
||||
// getters are built before the helper sources are emitted.
|
||||
const getters = this.bindings.map(({ exported, local }) =>
|
||||
`__dsh$def(exports,${JSON.stringify(exported)},()=>${this.bindingAccess(local)});`)
|
||||
if (this.bindings.length > 0) this.helper('def')
|
||||
for (const [name, source] of Object.entries(HELPER_SOURCE)) {
|
||||
if (this.helpers.has(name)) prologue.push(source)
|
||||
}
|
||||
for (const { exported, local } of this.bindings) {
|
||||
prologue.push(`__dsh$def(exports,${JSON.stringify(exported)},()=>${local});`)
|
||||
}
|
||||
prologue.push(...getters)
|
||||
|
||||
const sorted = [...this.edits].sort((left, right) => left.start - right.start || left.end - right.end)
|
||||
const render = (from: number, to: number): string => {
|
||||
@@ -521,6 +714,101 @@ function nameOf(node: Node): string {
|
||||
return node.type === 'Identifier' ? node.name as string : String(node.value)
|
||||
}
|
||||
|
||||
/** @returns The name an import or export specifier spells, as an identifier or a string literal. */
|
||||
function specifierName(node: Identifier | Literal): string {
|
||||
return node.type === 'Identifier' ? node.name : String(node.value)
|
||||
}
|
||||
|
||||
/**
|
||||
* Give every node the `range` that eslint-scope reads. The parse leaves ranges
|
||||
* out because only modules that need scope analysis use them.
|
||||
* @param node - A node or node array of the parsed module.
|
||||
*/
|
||||
function assignRanges(node: unknown): void {
|
||||
if (node === null || typeof node !== 'object') return
|
||||
if (Array.isArray(node)) {
|
||||
for (const child of node) assignRanges(child)
|
||||
return
|
||||
}
|
||||
const record = node as { type?: unknown; start?: unknown; end?: unknown; range?: readonly [unknown, unknown] }
|
||||
if (typeof record.type !== 'string') return
|
||||
record.range = [record.start, record.end]
|
||||
for (const value of Object.values(record)) assignRanges(value)
|
||||
}
|
||||
|
||||
/**
|
||||
* ESTree positions where an identifier names a declared or assigned binding.
|
||||
* Property values inside an object pattern are checked at the pattern.
|
||||
*
|
||||
* Together with the ObjectPattern check, this must recognize every ESTree
|
||||
* declaration and assignment form: a missed form lets a shadowing local skip
|
||||
* scope analysis and be rewritten as an import read. A parser upgrade that adds
|
||||
* a binding form must extend this list and the shadowing cases in
|
||||
* `transform-imports.spec.ts`.
|
||||
* @param parent - Parent of the identifier.
|
||||
* @param key - The parent property holding it.
|
||||
* @returns Whether the identifier declares or assigns a binding.
|
||||
*/
|
||||
function isBindingPosition(parent: Node, key: string | undefined): boolean {
|
||||
switch (parent.type) {
|
||||
case 'VariableDeclarator': return key === 'id'
|
||||
case 'FunctionDeclaration':
|
||||
case 'FunctionExpression':
|
||||
case 'ArrowFunctionExpression': return key === 'id' || key === 'params'
|
||||
case 'ClassDeclaration':
|
||||
case 'ClassExpression': return key === 'id'
|
||||
case 'CatchClause': return key === 'param'
|
||||
case 'ArrayPattern':
|
||||
case 'RestElement':
|
||||
case 'UpdateExpression': return true
|
||||
case 'AssignmentPattern':
|
||||
case 'AssignmentExpression':
|
||||
case 'ForInStatement':
|
||||
case 'ForOfStatement': return key === 'left'
|
||||
default: return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* ESTree positions where an identifier, outside a binding position, reads a
|
||||
* binding rather than naming a property, label, specifier, or meta property.
|
||||
* Positions not listed read a binding, so the binding positions must be ruled
|
||||
* out first (`isBindingPosition`).
|
||||
* @param parent - Parent of the identifier.
|
||||
* @param key - The parent property holding it.
|
||||
* @returns Whether the identifier reads a binding.
|
||||
*/
|
||||
function isReadPosition(parent: Node, key: string | undefined): boolean {
|
||||
switch (parent.type) {
|
||||
case 'MemberExpression': return key !== 'property' || parent.computed === true
|
||||
case 'Property':
|
||||
case 'MethodDefinition':
|
||||
case 'PropertyDefinition': return key !== 'key' || parent.computed === true
|
||||
case 'LabeledStatement':
|
||||
case 'BreakStatement':
|
||||
case 'ContinueStatement':
|
||||
case 'ImportSpecifier':
|
||||
case 'ImportDefaultSpecifier':
|
||||
case 'ImportNamespaceSpecifier':
|
||||
case 'ExportSpecifier':
|
||||
case 'ExportAllDeclaration':
|
||||
case 'MetaProperty': return false
|
||||
default: return true
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param parent - Parent of a statement.
|
||||
* @param key - The parent property holding it.
|
||||
* @returns Whether the statement sits in a statement list, where it can follow
|
||||
* a statement that ends without a semicolon.
|
||||
*/
|
||||
function inStatementList(parent: Node | undefined, key: string | undefined): boolean {
|
||||
if (parent === undefined) return false
|
||||
if (parent.type === 'SwitchCase') return key === 'consequent'
|
||||
return key === 'body' && (parent.type === 'Program' || parent.type === 'BlockStatement' || parent.type === 'StaticBlock')
|
||||
}
|
||||
|
||||
/** Every binding an exported declaration introduces, including patterns. */
|
||||
function declaredBindings(declaration: Node, fail: (detail: string) => never): Binding[] {
|
||||
if (declaration.type === 'FunctionDeclaration' || declaration.type === 'ClassDeclaration') {
|
||||
|
||||
@@ -0,0 +1,314 @@
|
||||
/**
|
||||
* Import bindings in lowered bodies (`src/compile/transform.ts`): a named or
|
||||
* default import is read through its module object at each reference, so a
|
||||
* module in an import cycle sees a binding once the exporting module has
|
||||
* initialized it, as ES modules do. Covers the cycle itself, live reads, the
|
||||
* operation each rewritten reference keeps, shadowing, and the refused forms.
|
||||
*/
|
||||
import { describe, expect, test } from 'vitest'
|
||||
import { lowerModuleSource } from '../../src/compile/transform.ts'
|
||||
import { WRAPPER_PARAMS } from '../../src/image-layout.ts'
|
||||
|
||||
/** @returns The lowered body of one probe module. */
|
||||
const transformModule = (source: string, path = 'probe.js'): string =>
|
||||
lowerModuleSource({ filename: path, source }).code
|
||||
|
||||
/**
|
||||
* Compile and run one lowered body with the loader's wrapper parameters.
|
||||
* @param code - Lowered body.
|
||||
* @param require - Module resolver the body's `require` calls reach.
|
||||
* @returns The populated `exports` object.
|
||||
*/
|
||||
function runBody(code: string, require: (specifier: string) => unknown = () => ({})): Record<string, unknown> {
|
||||
const module = { exports: {} as Record<string, unknown> }
|
||||
// eslint-disable-next-line @typescript-eslint/no-implied-eval -- the wrapper contract under test is a `new Function` body
|
||||
const factory = new Function(...WRAPPER_PARAMS, code) as (...args: unknown[]) => void
|
||||
factory(module.exports, require, module, '/vfs/probe.js', '/vfs', { url: 'file:///vfs/probe.js' }, undefined)
|
||||
return module.exports
|
||||
}
|
||||
|
||||
/**
|
||||
* Lower and load a module graph with the worker loader's CommonJS cycle rule:
|
||||
* a module required while it is still evaluating answers its partial exports.
|
||||
* @param sources - Module sources by request name.
|
||||
* @param entry - The module to load first.
|
||||
* @returns The entry module's exports.
|
||||
*/
|
||||
function loadGraph(sources: Record<string, string>, entry: string): Record<string, unknown> {
|
||||
const cache = new Map<string, { exports: Record<string, unknown> }>()
|
||||
const load = (request: string): Record<string, unknown> => {
|
||||
const name = request.replace(/^\.\//, '')
|
||||
const cached = cache.get(name)
|
||||
if (cached !== undefined) return cached.exports
|
||||
const source = sources[name]
|
||||
if (source === undefined) throw new Error(`no module named ${name}`)
|
||||
const module = { exports: {} as Record<string, unknown> }
|
||||
cache.set(name, module)
|
||||
// eslint-disable-next-line @typescript-eslint/no-implied-eval -- the wrapper contract under test is a `new Function` body
|
||||
const factory = new Function(...WRAPPER_PARAMS, transformModule(source, name)) as (...args: unknown[]) => void
|
||||
factory(module.exports, load, module, `/vfs/${name}`, '/vfs', { url: `file:///vfs/${name}` }, undefined)
|
||||
return module.exports
|
||||
}
|
||||
return load(entry)
|
||||
}
|
||||
|
||||
/** @returns The error message of a refused transform, or undefined when it succeeded. */
|
||||
const refusal = (source: string): string | undefined => {
|
||||
try {
|
||||
transformModule(source)
|
||||
return undefined
|
||||
} catch (reason) {
|
||||
return (reason as Error).message
|
||||
}
|
||||
}
|
||||
|
||||
describe('import cycles', () => {
|
||||
test('a module reads an import after the exporting module initializes it', () => {
|
||||
// core.js starts first and requires util.js, which requires core.js back while
|
||||
// core.js has not yet reached `globalConfig` (the shape of zod's core/util pair).
|
||||
const core = loadGraph({
|
||||
'core.js': "import { members } from './util.js'\nexport function make() { return members() }\nexport const globalConfig = { jitless: true }\n",
|
||||
'util.js': "import { globalConfig } from './core.js'\nexport function members() { return globalConfig.jitless }\n",
|
||||
}, 'core.js')
|
||||
expect((core.make as () => boolean)()).toBe(true)
|
||||
})
|
||||
|
||||
test('a default import reads the exporting module after it finishes', () => {
|
||||
const entry = loadGraph({
|
||||
'a.js': "import B from './b.js'\nexport const read = () => B()\nexport default function A() { return 'a' }\n",
|
||||
'b.js': "import A from './a.js'\nexport default function B() { return A() + 'b' }\n",
|
||||
}, 'a.js')
|
||||
expect((entry.read as () => string)()).toBe('ab')
|
||||
})
|
||||
})
|
||||
|
||||
describe('live reads', () => {
|
||||
test('a named import observes a later change on its module', () => {
|
||||
const module: Record<string, unknown> = { counter: 1 }
|
||||
const exports = runBody(transformModule("import { counter } from 'p'\nexport const read = () => counter\n"), () => module)
|
||||
module.counter = 2
|
||||
expect((exports.read as () => number)()).toBe(2)
|
||||
})
|
||||
|
||||
test('a default import observes a later change on its module', () => {
|
||||
const module: Record<string, unknown> = { __esModule: true, default: 1 }
|
||||
const exports = runBody(transformModule("import value from 'p'\nexport const read = () => value\n"), () => module)
|
||||
module.default = 2
|
||||
expect((exports.read as () => number)()).toBe(2)
|
||||
})
|
||||
|
||||
test('re-exporting an import through a local export clause stays live', () => {
|
||||
const module: Record<string, unknown> = { a: 1 }
|
||||
const exports = runBody(transformModule("import { a } from 'p'\nexport { a, a as alias }\n"), () => module)
|
||||
module.a = 2
|
||||
expect([exports.a, exports.alias]).toEqual([2, 2])
|
||||
})
|
||||
|
||||
test('a default import re-exported through a local export clause stays live', () => {
|
||||
const module: Record<string, unknown> = { __esModule: true, default: 1 }
|
||||
const exports = runBody(transformModule("import value from 'p'\nexport { value }\n"), () => module)
|
||||
module.default = 2
|
||||
expect(exports.value).toBe(2)
|
||||
})
|
||||
})
|
||||
|
||||
describe('rewritten references keep their operation', () => {
|
||||
const receiver = function (this: unknown): unknown { return this }
|
||||
|
||||
test('calling a named import passes no receiver', () => {
|
||||
const exports = runBody(transformModule("import { f } from 'p'\nexport const seen = f()\n"), () => ({ f: receiver }))
|
||||
expect(exports.seen).toBeUndefined()
|
||||
})
|
||||
|
||||
test('an optional call of a named import passes no receiver', () => {
|
||||
const exports = runBody(transformModule("import { f } from 'p'\nexport const seen = f?.()\n"), () => ({ f: receiver }))
|
||||
expect(exports.seen).toBeUndefined()
|
||||
})
|
||||
|
||||
test('a named import used as a template tag passes no receiver', () => {
|
||||
const exports = runBody(transformModule("import { f } from 'p'\nexport const seen = f`x`\n"), () => ({ f: receiver }))
|
||||
expect(exports.seen).toBeUndefined()
|
||||
})
|
||||
|
||||
test('new constructs named and default imports, including a member of a default import', () => {
|
||||
class Named { readonly kind = 'named' }
|
||||
class Inner { readonly kind = 'inner' }
|
||||
class Default { static readonly Inner = Inner; readonly kind = 'default' }
|
||||
const code = transformModule([
|
||||
"import D, { Named } from 'p'",
|
||||
'export const made = [new Named().kind, new D().kind, new D.Inner().kind, new D, new Named]',
|
||||
].join('\n'))
|
||||
const made = runBody(code, () => ({ __esModule: true, default: Default, Named })).made as unknown[]
|
||||
expect(made.slice(0, 3)).toEqual(['named', 'default', 'inner'])
|
||||
expect([made[3] instanceof Default, made[4] instanceof Named]).toEqual([true, true])
|
||||
})
|
||||
|
||||
test('a shorthand property keeps its key and reads the import', () => {
|
||||
const exports = runBody(transformModule("import { a } from 'p'\nexport const object = { a }\n"), () => ({ a: 1 }))
|
||||
expect(exports.object).toEqual({ a: 1 })
|
||||
})
|
||||
|
||||
test('computed keys, class heritage, typeof, spread, and default export read the import', () => {
|
||||
class Base { readonly base = true }
|
||||
const code = transformModule([
|
||||
"import { key, Base, list } from 'p'",
|
||||
'export const object = { [key]: typeof Base, ...list }',
|
||||
'export class Derived extends Base {}',
|
||||
'export default key',
|
||||
].join('\n'))
|
||||
const exports = runBody(code, () => ({ key: 'k', Base, list: ['z'] }))
|
||||
expect(exports.object).toEqual({ k: 'function', 0: 'z' })
|
||||
expect(new (exports.Derived as new () => { base: boolean })().base).toBe(true)
|
||||
expect(exports.default).toBe('k')
|
||||
})
|
||||
|
||||
test('a call at the start of a statement does not extend a preceding statement without a semicolon', () => {
|
||||
let calls = 0
|
||||
const code = transformModule("import d, { f } from 'p'\nexport const a = 1\nf()\nexport const b = 2\nd.bump()\n")
|
||||
const exports = runBody(code, () => ({ __esModule: true, default: { bump: () => { calls += 1 } }, f: () => { calls += 1 } }))
|
||||
expect([exports.a, exports.b, calls]).toEqual([1, 2, 2])
|
||||
})
|
||||
|
||||
test('a call as the body of an if statement stays conditional', () => {
|
||||
let calls = 0
|
||||
runBody(transformModule("import { f } from 'p'\nif (false) f()\n"), () => ({ f: () => { calls += 1 } }))
|
||||
expect(calls).toBe(0)
|
||||
})
|
||||
|
||||
test('calling a default import at the start of a statement does not extend the preceding statement', () => {
|
||||
let calls = 0
|
||||
const exports = runBody(
|
||||
transformModule("import d from 'p'\nexport const a = 1\nd()\n"),
|
||||
() => ({ __esModule: true, default: () => { calls += 1 } }),
|
||||
)
|
||||
expect([exports.a, calls]).toEqual([1, 1])
|
||||
})
|
||||
|
||||
test('references inside rewritten suspension points are rewritten too', async () => {
|
||||
const code = transformModule([
|
||||
"import { source, value } from 'p'",
|
||||
'export const run = async () => { const seen = []; for await (const item of source) seen.push(item + await value); return seen }',
|
||||
].join('\n'))
|
||||
const als = {
|
||||
pause: (awaited: unknown) => Promise.resolve(awaited).then(settled => ({ ok: true, value: settled })),
|
||||
resume: (token: { value: unknown }) => token.value,
|
||||
iterator: (iterable: Iterable<number>) => {
|
||||
const inner = iterable[Symbol.iterator]()
|
||||
return { next: () => Promise.resolve(inner.next()) }
|
||||
},
|
||||
close: () => Promise.resolve(),
|
||||
}
|
||||
const module = { exports: {} as Record<string, unknown> }
|
||||
// eslint-disable-next-line @typescript-eslint/no-implied-eval -- the wrapper contract under test is a `new Function` body
|
||||
const factory = new Function(...WRAPPER_PARAMS, code) as (...args: unknown[]) => void
|
||||
factory(module.exports, () => ({ source: [1, 2], value: 10 }), module, '/vfs/probe.js', '/vfs', { url: 'file:///vfs/probe.js' }, als)
|
||||
expect(await (module.exports.run as () => Promise<number[]>)()).toEqual([11, 12])
|
||||
})
|
||||
})
|
||||
|
||||
describe('emitted import statements', () => {
|
||||
test('named and default imports declare no locals; namespace imports keep theirs', () => {
|
||||
const code = transformModule("import d, { a, b as c } from 'p'\nimport * as ns from 'q'\nexport const out = [d, a, c, ns]\n")
|
||||
expect(code).toContain('const __dsh$m1=require("p");')
|
||||
expect(code).toContain('const ns=__dsh$ns(__dsh$m2);')
|
||||
expect(code).not.toMatch(/const (d|a|c)=/)
|
||||
expect(code).toContain('__dsh$m1["b"]')
|
||||
})
|
||||
|
||||
test('a reference ahead of its import declaration reads the same module', () => {
|
||||
const exports = runBody(transformModule("export const read = () => a\nimport { a } from 'p'\n"), () => ({ a: 1 }))
|
||||
expect((exports.read as () => number)()).toBe(1)
|
||||
})
|
||||
|
||||
test('a same-named local elsewhere changes nothing in the lines that read the import', () => {
|
||||
// The parameter `a` declares an import's name, which sends the module
|
||||
// through scope analysis; the shared lines must lower identically.
|
||||
const shared = "import d, { a, b } from 'p'\nexport const read = () => [a, b(), { a }, d.x]\nb``\n"
|
||||
const plain = transformModule(shared).split('\n')
|
||||
const shadowed = transformModule(`${shared}function unrelated(a) { return a }\n`).split('\n')
|
||||
expect(shadowed.slice(0, 3)).toEqual(plain.slice(0, 3))
|
||||
expect(shadowed[3]).toBe('function unrelated(a) { return a }')
|
||||
})
|
||||
})
|
||||
|
||||
/**
|
||||
* Cases where a name `x` is also imported: `probe(() => x)` records whether
|
||||
* the reference reached the import or a local binding (including one still in
|
||||
* its temporal dead zone). Expected values follow ES module scoping; each case
|
||||
* gives the same answer under Node's native ES module loader.
|
||||
*/
|
||||
const SHADOWING_CASES: ReadonlyArray<readonly [name: string, body: string, expected: 'import' | 'local']> = [
|
||||
['plain reference', 'probe(() => x)', 'import'],
|
||||
['parameter', 'function f(x){ probe(() => x) } f(1)', 'local'],
|
||||
['let later in the same block', '{ probe(() => x); let x = 1 }', 'local'],
|
||||
['var hoisted from a nested block', 'function f(){ probe(() => x); if (1) { var x = 1 } } f()', 'local'],
|
||||
['function declared in a block, read outside', '{ function x(){} } probe(() => x)', 'import'],
|
||||
['class expression name inside the class', 'const C = class x { static m(){ probe(() => x) } }; C.m()', 'local'],
|
||||
['named function expression inside itself', 'const g = function x(){ probe(() => x) }; g()', 'local'],
|
||||
['catch parameter', 'try { throw 1 } catch (x) { probe(() => x) }', 'local'],
|
||||
['destructured catch parameter', 'try { throw { x: 1 } } catch ({ x }) { probe(() => x) }', 'local'],
|
||||
['for-of let inside the loop', 'for (let x of [1]) probe(() => x)', 'local'],
|
||||
['for-of let after the loop', 'for (let x of [1]) {} probe(() => x)', 'import'],
|
||||
['let in a switch case', 'switch (1) { case 1: let x = 1; probe(() => x) }', 'local'],
|
||||
['parameter default naming a later parameter', 'const h = (a = probe(() => x), x) => 0; h()', 'local'],
|
||||
['let in a class static block', 'class K { static { let x = 1; probe(() => x) } }', 'local'],
|
||||
['destructured parameter', 'function f({ a: [x] }) { probe(() => x) } f({ a: [1] })', 'local'],
|
||||
['function declared later in the body', 'function f(){ probe(() => x); function x(){} } f()', 'local'],
|
||||
['label of the same name', 'x: for (;;) { probe(() => x); break x }', 'import'],
|
||||
['class declared later in the block', '{ probe(() => x); class x {} }', 'local'],
|
||||
['parameter of a sibling function', 'function a(x){} function b(){ probe(() => x) } b()', 'import'],
|
||||
['parameter default beside a body var', 'function f(a = probe(() => x)) { var x = 1 } f()', 'import'],
|
||||
['var in a class static block', 'class K { static { var x = 1; probe(() => x) } }', 'local'],
|
||||
['let in a static block, read outside the class', 'class K { static { let x = 1 } }\nprobe(() => x)', 'import'],
|
||||
['rest parameter', 'function f(...x) { probe(() => x) } f(1)', 'local'],
|
||||
['rest element in array destructuring', '{ const [...x] = [1]; probe(() => x) }', 'local'],
|
||||
['rest property in object destructuring', '{ const { ...x } = {}; probe(() => x) }', 'local'],
|
||||
['parameter with a default value', 'function f(x = 1) { probe(() => x) } f()', 'local'],
|
||||
['destructured property with a default value', '{ const { x = 1 } = {}; probe(() => x) }', 'local'],
|
||||
]
|
||||
|
||||
describe('a reference reads the import exactly where ES module scoping resolves to it', () => {
|
||||
const imported = Symbol('imported')
|
||||
test.each(SHADOWING_CASES)('%s', (_name, body, expected) => {
|
||||
const seen: string[] = []
|
||||
const probe = (read: () => unknown): void => {
|
||||
try {
|
||||
seen.push(read() === imported ? 'import' : 'local')
|
||||
} catch (reason) {
|
||||
if (!(reason instanceof ReferenceError)) throw reason
|
||||
seen.push('local')
|
||||
}
|
||||
}
|
||||
const code = transformModule(`import { x } from 'm'\nimport { probe } from 'probe'\n${body}\n`)
|
||||
runBody(code, specifier => (specifier === 'm' ? { x: imported } : { probe }))
|
||||
expect(seen).toEqual([expected])
|
||||
})
|
||||
})
|
||||
|
||||
describe('refused forms', () => {
|
||||
test.each([
|
||||
['assigning a named import', "import { x } from 'p'\nx = 1\n"],
|
||||
['updating a named import', "import { x } from 'p'\nexport const f = () => x++\n"],
|
||||
['destructuring into a named import', "import { x } from 'p'\n;({ x } = {})\n"],
|
||||
['assigning a default import', "import d from 'p'\nexport const f = () => { d = 1 }\n"],
|
||||
['assigning a named import in a for-of head', "import { x } from 'p'\nfor (x of []) {}\n"],
|
||||
])('%s fails the transform', (_name, source) => {
|
||||
expect(refusal(source)).toMatch(/the import \w+ cannot be assigned \(probe\.js:\d+\)/)
|
||||
})
|
||||
|
||||
test('a direct eval in a module with named imports fails the transform', () => {
|
||||
expect(refusal("import { x } from 'p'\nexport const f = () => eval('x')\n")).toContain('direct eval')
|
||||
})
|
||||
|
||||
test('an optional call of eval is indirect and still lowers', () => {
|
||||
expect(refusal("import { x } from 'p'\nexport const f = () => [x, eval?.('1')]\n")).toBeUndefined()
|
||||
})
|
||||
|
||||
test('a direct eval in a module without named or default imports still lowers', () => {
|
||||
expect(refusal("import * as ns from 'p'\nexport const f = () => eval('ns')\n")).toBeUndefined()
|
||||
})
|
||||
|
||||
test('a declaration using the reserved prefix fails the transform', () => {
|
||||
expect(refusal("import { x } from 'p'\nfunction f() { const __dsh$m1 = x }\n")).toContain('reserved __dsh$ prefix')
|
||||
})
|
||||
})
|
||||
@@ -192,8 +192,8 @@ check(
|
||||
}
|
||||
|
||||
{
|
||||
// Named imports are snapshots (CommonJS destructuring semantics), which is the
|
||||
// documented, accepted divergence from ESM live bindings on the import side.
|
||||
// Named imports read the required module at each reference; cycles, live
|
||||
// reads, and shadowing are covered in transform-imports.spec.ts.
|
||||
const code = transformModule("import { a, b as c } from 'p'\nexport const out = [a, c]\n", 'probe.js')
|
||||
parsesAsScript('named imports', code)
|
||||
const exports = runBody(code, () => ({ a: 1, b: 2 }))
|
||||
|
||||
Generated
+9
@@ -11653,6 +11653,12 @@ importers:
|
||||
buffer:
|
||||
specifier: ^6.0.3
|
||||
version: 6.0.3
|
||||
eslint-scope:
|
||||
specifier: ^9.1.2
|
||||
version: 9.1.2
|
||||
eslint-visitor-keys:
|
||||
specifier: ^5.0.1
|
||||
version: 5.0.1
|
||||
picomatch:
|
||||
specifier: ^4.0.7
|
||||
version: 4.0.7
|
||||
@@ -11714,6 +11720,9 @@ importers:
|
||||
'@deepseek-ai/node-addon-system':
|
||||
specifier: workspace:~
|
||||
version: link:../../../native/system/packages/entry
|
||||
'@types/estree':
|
||||
specifier: ^1.0.9
|
||||
version: 1.0.9
|
||||
'@types/picomatch':
|
||||
specifier: ^4.0.3
|
||||
version: 4.0.3
|
||||
|
||||
@@ -1139,9 +1139,6 @@
|
||||
"packages/experimental/webworker-packer/tests/preview.spec.ts": {
|
||||
"cdd46c8ceda6d5707bc2abfba329351ba06bcc030b314a3c799d72a4971bd5a4": 1
|
||||
},
|
||||
"packages/experimental/webworker-runtime/src/compile/transform.ts": {
|
||||
"48bb7e26a882fa7a44a4175f843d20846606976c7d6fdaf67c8a08dd8830b95e": 1
|
||||
},
|
||||
"packages/experimental/webworker-runtime/src/node/builtin_modules/implemented/stream.ts": {
|
||||
"8d4298961ca8095e069df7c940bd5082e0a90f76a24f517d5f547566e0631b6c": 1
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user