feat(gates): reject module-scope await in package sources

Package modules that other code imports must not suspend evaluation at module scope. A syntax-aware gate scans packages/*/*/src and excludes only executables (a shebang or a package bin target). The SenseVoice worker, the SSH helper entry, and the Inspector worker entry now run an explicit main().
This commit is contained in:
imccyu
2026-10-15 03:56:25 +08:00
parent 7ba5e3a01f
commit b77cc1c0f2
15 changed files with 733 additions and 34 deletions
@@ -0,0 +1,22 @@
# Bilingual-pair consistency record for 2026-10-14-no-top-level-await-in-packages.md (docs/i18n/README.md): per heading
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
# After editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-10-14-no-top-level-await-in-packages.md
/agent-note-package-sources-must-not-use-top-level-await:
en: d7409565b91b608c
zh: 855807c10ac7e44a
/agent-note-package-sources-must-not-use-top-level-await/problem:
en: 1735c60cec478b06
zh: 85ed83b2b804b39e
/agent-note-package-sources-must-not-use-top-level-await/decision:
en: 7459c979b58be927
zh: 569c68e1476f0129
/agent-note-package-sources-must-not-use-top-level-await/alternatives-considered:
en: 5f19be0393f39abc
zh: ed0414b54eb17678
/agent-note-package-sources-must-not-use-top-level-await/consequences:
en: d609ff0b1ce0c99a
zh: d60aa801ea72c2dd
/agent-note-package-sources-must-not-use-top-level-await/testing:
en: 55adbea10005be4b
zh: 8c95636d2a0fdad4
@@ -0,0 +1,45 @@
# Agent Note: Package sources must not use top-level await
Status: implemented
English | [中文](2026-10-14-no-top-level-await-in-packages.zh.md)
## Problem
The child-process entry `src/worker.ts` of [speech-to-text-sensevoice](../../../../packages/experimental/speech-to-text-sensevoice/README.md) ran `await startRecognitionServer(...)` at module scope. Top-level await makes module evaluation asynchronous: an importer's evaluation waits on it, and an entry failure becomes a module-loading failure instead of an error the entry handles explicitly. The same file is also the package's `./worker` export, and the browser Preview's module loader wraps module bodies as CommonJS, so it refuses to pack top-level await, which kept voice-input-bundle out of the image. No repository gate prevented this pattern; a repository-wide scan found two more cases outside bins: ssh's `src/helper-entry.ts` (the target of the `./helper` export) and inspector's `src/worker/entry.ts` (a worker_threads entry).
## Decision
- Entry modules use an explicit `async function main()`:
- sensevoice: a failure before readiness writes only the error message to stderr and exits with code 1 once the message is flushed (`process.exit(1)` in the write callback, because a loaded native recognizer would otherwise keep the process alive); stdout carries only one `{"port":N}` line, which the Host parses as readiness.
- ssh helper: `main()` holds the helper's try/catch/finally and signal handling; every failure is reported on stderr with the `dsh-ssh-sandbox:` prefix and exit code 127.
- inspector worker: the synchronous startup stays at module scope, the asynchronous part runs in `main()`, and a failure sends the failure frame before stopping.
- Gate [verify-no-top-level-await](../../../../scripts/verify-no-top-level-await.ts):
- **Scope**: `.ts` and `.tsx` files under `packages/*/*/src/**`.
- **Rule**: based on the TypeScript AST. An `await` expression, `for await`, or `await using` whose nearest function body is the module is a violation, including occurrences in top-level if statements, blocks, try statements, loops, switch statements, conditional expressions, export initializers, and computed member names. Occurrences in functions, methods, constructors, accessors, parameter defaults, and class static blocks are not violations.
- **Exclusions**: only bins are excluded, judged from declarations and syntax alone: the source file starts with `#!`, or it is the `src/<path>.ts|.tsx` that corresponds to a `lib/<path>.js` named by package.json `bin`. A process executes a bin directly, and no other module imports it. The current exclusions are the `src/bin.ts` files of webworker-packer and llm-mock-server, and a spec pins the list.
- **Corpus guard**: the gate fails when the corpus is empty, contains no `.tsx`, or completely omits a `src/` directory that contains TS.
- **Registration**: a shared hygiene gate in `run-gates`, running in the static lane and the hygiene aggregate. The rule text lives in the [package-surface defensive patterns](../../../../docs/defensive-patterns/package-surface.md).
## Alternatives considered
**Rely only on the webworker packer's refusal.** The packer checks only modules that enter the image, so packages outside the image could still introduce top-level await.
**Implement it as a lint rule.** oxlint has no existing rule that expresses "module scope" precisely. A syntax-level script gate is more controllable, and tests can pin every included and excluded form.
**Forbid it in bins too.** A process executes a bin directly and nothing imports it, so its top-level await does not affect loading any module.
## Consequences
Every importable package source evaluates synchronously, including packages that never enter the Preview image, and each entry handles its own startup failure; voice-input-bundle enters the Preview image.
The trade-off costs:
- Third-party packages and `apps/` are outside the scope; the page bootstrap script `apps/web/src/preview.ts` uses top-level await.
- sensevoice startup failure diagnostics carry only the error message, without a stack trace.
## Testing
- [verify-no-top-level-await.spec.ts](../../../../scripts/verify-no-top-level-await.spec.ts) pins every included and excluded form, the corpus guards, and the current exclusion list; [run-gates.spec.ts](../../../../scripts/run-gates.spec.ts) pins the gate's registration. With the asynchronous startup of the three entries above at module scope, the gate reports 4 cases (1 in sensevoice, 1 in ssh, 2 in inspector) and fails.
- [worker-entry.spec.ts](../../../../packages/experimental/speech-to-text-sensevoice/tests/worker-entry.spec.ts) launches the sensevoice entry and pins that a failure before readiness exits with code 1, leaves stdout empty, and reports the reason on stderr without a stack trace.
- [helper-boundaries.spec.ts](../../../../packages/ssh/ssh/tests/helper-boundaries.spec.ts) launches the ssh helper entry, and [integration.host.spec.ts](../../../../packages/experimental/inspector/tests/integration.host.spec.ts) starts the inspector Worker through `startInspector`; no spec drives the inspector worker entry's startup-failure frame.
@@ -0,0 +1,45 @@
# Agent Note: 包源码禁止顶层 await
Status: implemented
[English](2026-10-14-no-top-level-await-in-packages.md) | 中文
## 问题
[speech-to-text-sensevoice](../../../../packages/experimental/speech-to-text-sensevoice/README.zh.md) 的子进程入口 `src/worker.ts` 曾在模块顶层 `await startRecognitionServer(...)`。顶层 await 让模块求值变成异步:导入方的求值被它挂住,入口的失败也变成模块加载失败,而不是入口自己显式处理的错误。这个文件同时是包的 `./worker` 导出,浏览器 Preview 的模块加载器按 CommonJS 包装模块体,遇到顶层 await 会拒绝打包,voice-input-bundle 因此无法进入镜像。仓库里当时没有任何门禁阻止这种写法;全仓扫描时,除 bin 外还有另外两处:ssh 的 `src/helper-entry.ts`(`./helper` 导出的目标)和 inspector 的 `src/worker/entry.ts`(worker_threads 入口)。
## 决策
- 入口模块用显式 `async function main()`:
- sensevoice:就绪前失败时只把错误消息写到 stderr,消息写完后以退出码 1 退出(在写入回调里调用 `process.exit(1)`,否则已加载的原生识别器会让进程一直存活);stdout 只有一行 `{"port":N}`,Host 把它解析为就绪信号。
- ssh helper:`main()` 承载 helper 的 try/catch/finally 和信号处理;每个失败都以 `dsh-ssh-sandbox:` 前缀写到 stderr,退出码为 127。
- inspector worker:同步启动部分留在模块顶层,异步部分在 `main()` 里运行,失败时先发 failure 帧再 stop。
- 门禁 [verify-no-top-level-await](../../../../scripts/verify-no-top-level-await.ts):
- **范围**:`packages/*/*/src/**` 下的 `.ts` 和 `.tsx`。
- **判定**:基于 TypeScript AST。最近的函数体是模块本身的 `await` 表达式、`for await`、`await using` 都算违规,包括顶层 if、块、try、循环、switch、条件表达式、export 初始化和计算成员名里的。函数、方法、构造器、访问器、参数默认值、class static block 里的不算。
- **排除**:只排除 bin,依据只取声明和语法:源文件以 `#!` 开头,或者是 package.json `bin` 指向的 `lib/<path>.js` 对应的 `src/<path>.ts|.tsx`。bin 由进程直接执行,不会被其他模块导入。当前排除 webworker-packer 和 llm-mock-server 的两个 `src/bin.ts`,清单由 spec 锁定。
- **语料防护**:语料为空、没有 `.tsx`、或某个含 TS 的 `src/` 目录完全没进语料时,门禁报错。
- **注册**:`run-gates` 的共享卫生门禁,跑在 static lane 和 hygiene 聚合里。规则正文写在 [package-surface 防御模式](../../../../docs/defensive-patterns/package-surface.zh.md)。
## 考虑过的替代方案
**只靠 webworker packer 的拒绝来把关。** packer 只检查进入镜像的模块,镜像外的包仍可能引入顶层 await。
**用 lint 规则实现。** oxlint 没有现成规则能精确表达“模块顶层”。语法级脚本门禁更可控,也能用测试锁定每一种纳入和排除的形式。
**连 bin 一起禁止。** bin 由进程直接执行,不被导入,顶层 await 不影响任何模块的加载。
## 后果
每个可被导入的包源码都同步求值,包括从不进入 Preview 镜像的包;每个入口自己处理启动失败;voice-input-bundle 能进入 Preview 镜像。
这一取舍的代价:
- 第三方包和 `apps/` 不在范围内;页面引导脚本 `apps/web/src/preview.ts` 使用顶层 await。
- sensevoice 启动失败的诊断只有错误消息,不带堆栈。
## 测试
- [verify-no-top-level-await.spec.ts](../../../../scripts/verify-no-top-level-await.spec.ts) 锁定每一种纳入和排除的形式、语料防护和当前的排除清单;[run-gates.spec.ts](../../../../scripts/run-gates.spec.ts) 锁定门禁的注册。若上述三个入口的异步启动放在模块顶层,门禁报出 4 处(sensevoice 1 处、ssh 1 处、inspector 2 处)并失败。
- [worker-entry.spec.ts](../../../../packages/experimental/speech-to-text-sensevoice/tests/worker-entry.spec.ts) 启动 sensevoice 入口,锁定就绪前失败时退出码为 1、stdout 为空、stderr 给出原因且不带堆栈。
- [helper-boundaries.spec.ts](../../../../packages/ssh/ssh/tests/helper-boundaries.spec.ts) 启动 ssh helper 入口,[integration.host.spec.ts](../../../../packages/experimental/inspector/tests/integration.host.spec.ts) 经 `startInspector` 启动 inspector Worker;没有 spec 触发 inspector worker 入口的启动失败帧。
@@ -23,3 +23,6 @@
/package-surface-rules/a-new-resolve-hook-is-refused-outright:
en: b420ae3e228432d5
zh: 0c49c4a9c3a5262c
/package-surface-rules/package-modules-do-not-await-at-module-scope:
en: 29a58b3f804be042
zh: 8a83a21366dac897
@@ -222,3 +222,10 @@ const previous = hooks.resolve
hooks.resolve = (specifier, context, next) =>
pinned.has(specifier) ? pinned.get(specifier) : previous(specifier, context, next)
```
## Package modules do not await at module scope
A package source module starts its asynchronous work inside a function it calls, such as an explicit `main()` that handles its own rejection, and never uses `await`, `for await`, or `await using` at module scope: a module-scope wait makes every importer's evaluation wait, turns a rejection into a module-evaluation failure, and cannot be lowered to the synchronous CommonJS wrapper of the browser-worker image.
- Judged by [verify-no-top-level-await.ts](../../scripts/verify-no-top-level-await.ts), run as `pnpm run verify-no-top-level-await` and by `scripts/run-gates.ts` as the `no-top-level-await` gate; it parses every `packages/*/*/src/**/*.{ts,tsx}` file and excludes executables by declaration only: a source that starts with `#!`, or the source of a `lib/` output that the package's `bin` field names.
- Exception: none.
@@ -222,3 +222,10 @@ const previous = hooks.resolve
hooks.resolve = (specifier, context, next) =>
pinned.has(specifier) ? pinned.get(specifier) : previous(specifier, context, next)
```
## 包模块不在模块顶层 await
包源码模块的异步工作在它调用的函数里启动(例如自己处理 rejection 的显式 `main()`),模块顶层不使用 `await`、`for await` 或 `await using`:模块顶层的等待会让每个导入方的求值跟着等待,把 rejection 变成模块求值失败,而且无法降级成浏览器 Worker 镜像使用的同步 CommonJS 包装。
- 判定依据:[verify-no-top-level-await.ts](../../scripts/verify-no-top-level-await.ts),以 `pnpm run verify-no-top-level-await` 运行,也由 `scripts/run-gates.ts` 作为 `no-top-level-await` 门禁运行;它解析每个 `packages/*/*/src/**/*.{ts,tsx}` 文件,只按声明排除可执行入口:以 `#!` 开头的源码,或包的 `bin` 字段指向的 `lib/` 产物所对应的源码。
- 例外:无。
+1
View File
@@ -149,6 +149,7 @@
"verify-client-ui-i18n": "tsx scripts/verify-client-ui-i18n.ts",
"verify-client-route-resolution": "tsx scripts/verify-client-route-resolution.ts",
"verify-no-bare-dispatcher": "tsx scripts/verify-no-bare-dispatcher.ts",
"verify-no-top-level-await": "tsx scripts/verify-no-top-level-await.ts",
"verify-plugin-record-callers": "tsx scripts/verify-plugin-record-callers.ts",
"verify-cordis-config": "tsx scripts/verify-cordis-config.ts",
"rescope-vendor": "tsx scripts/rescope-vendor.ts",
@@ -43,13 +43,18 @@ controlPort.on('message', (message: unknown) => {
}
})
try {
runtime = await startInspectorWorker(boot)
controlPort.postMessage({ type: 'ready', ...runtime.endpoint } satisfies InspectorWorkerControl)
} catch (error) {
controlPort.postMessage({
type: 'failure',
message: error instanceof Error ? error.message : String(error),
} satisfies InspectorWorkerControl)
await stop()
/** Start the Worker server and report readiness, or report the failure and stop. */
async function main(): Promise<void> {
try {
runtime = await startInspectorWorker(boot)
controlPort.postMessage({ type: 'ready', ...runtime.endpoint } satisfies InspectorWorkerControl)
} catch (error) {
controlPort.postMessage({
type: 'failure',
message: error instanceof Error ? error.message : String(error),
} satisfies InspectorWorkerControl)
await stop()
}
}
void main()
@@ -1,13 +1,27 @@
/** Private child entry; the Host owns termination, and stdout carries only readiness. */
/**
* Private child process entry: load the native recognizer, listen on loopback, then print
* `{"port":N}` and a newline as the only stdout output. The Host passes the configuration and
* model paths as JSON in `argv[2]` and the bearer secret in `DSH_SPEECH_TOKEN`, which is removed
* from the environment once read. A failure before readiness writes only its message to stderr,
* which the Host includes in the preparation error, and exits with code 1. After readiness the
* loopback listener keeps the process alive until the Host terminates it.
*/
import { z } from 'zod'
import { Config } from './config.ts'
import { createTranscriber } from './inference.ts'
import { startRecognitionServer } from './process-server.ts'
const raw: unknown = JSON.parse(z.string().parse(process.argv[2]))
const paths = z.object({ model: z.string().min(1), tokens: z.string().min(1), vad: z.string().min(1) }).parse(raw)
const config = Object.assign(Config(z.record(z.string(), z.unknown()).parse(raw)), paths)
const token = z.string().regex(/^[a-f0-9]{64}$/).parse(process.env.DSH_SPEECH_TOKEN)
delete process.env.DSH_SPEECH_TOKEN
const { port } = await startRecognitionServer(token, config.maxAudioBytes, createTranscriber(config))
process.stdout.write(`${JSON.stringify({ port })}\n`)
async function main(): Promise<void> {
const raw: unknown = JSON.parse(z.string().parse(process.argv[2]))
const paths = z.object({ model: z.string().min(1), tokens: z.string().min(1), vad: z.string().min(1) }).parse(raw)
const config = Object.assign(Config(z.record(z.string(), z.unknown()).parse(raw)), paths)
const token = z.string().regex(/^[a-f0-9]{64}$/).parse(process.env.DSH_SPEECH_TOKEN)
delete process.env.DSH_SPEECH_TOKEN
const { port } = await startRecognitionServer(token, config.maxAudioBytes, createTranscriber(config))
process.stdout.write(`${JSON.stringify({ port })}\n`)
}
main().catch((error: unknown) => {
// Exit once the message is flushed: a loaded native recognizer would otherwise keep the process alive.
process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`, () => { process.exit(1) })
})
@@ -0,0 +1,57 @@
/**
* The private worker entry run as the Host launches source workers: plain Node with the tsx ESM hook and
* the repository paths map. Failures before readiness must leave stdout empty, because the Host parses its
* first line as readiness, and must exit 1 with only the failure message on stderr, because the Host shows
* that text in the preparation error.
*/
import { spawn } from 'node:child_process'
import { fileURLToPath } from 'node:url'
import { expect, it } from 'vitest'
const ENTRY = fileURLToPath(new URL('../src/worker.ts', import.meta.url))
const TSCONFIG = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url))
const TOKEN = 'a'.repeat(64)
interface Exit {
readonly code: number | null
readonly stdout: string
readonly stderr: string
}
/** Launch the entry and collect its complete output; the child is always awaited to exit. */
async function launch(argument: string | undefined, token: string | undefined): Promise<Exit> {
const env: NodeJS.ProcessEnv = { ...process.env, TSX_TSCONFIG_PATH: TSCONFIG }
delete env.DSH_SPEECH_TOKEN
if (token !== undefined) env.DSH_SPEECH_TOKEN = token
const child = spawn(process.execPath, ['--import', import.meta.resolve('tsx/esm'), ENTRY, ...argument === undefined ? [] : [argument]], {
env, stdio: ['ignore', 'pipe', 'pipe'],
})
let stdout = ''
let stderr = ''
child.stdout.setEncoding('utf8').on('data', (chunk: string) => { stdout += chunk })
child.stderr.setEncoding('utf8').on('data', (chunk: string) => { stderr += chunk })
const code = await new Promise<number | null>((resolve, reject) => {
child.once('error', reject)
child.once('close', resolve)
})
return { code, stdout, stderr }
}
const LAUNCH = { dataRoot: '/nonexistent/dsh-speech', model: 'model.onnx', tokens: 'tokens.txt', vad: 'vad.onnx' }
it.each([
['a missing launch argument', undefined, TOKEN, 'invalid_type'],
['a malformed launch argument', '{', TOKEN, 'JSON'],
['launch JSON without model paths', JSON.stringify({ dataRoot: LAUNCH.dataRoot }), TOKEN, '"model"'],
['launch JSON without a data root', JSON.stringify({ ...LAUNCH, dataRoot: undefined }), TOKEN, 'dataRoot'],
['a missing bearer token', JSON.stringify(LAUNCH), undefined, 'invalid_type'],
['a malformed bearer token', JSON.stringify(LAUNCH), 'not-hex', '[a-f0-9]{64}'],
])('exits 1 before readiness on %s, reporting only the failure message', async (_case, argument, token, reason) => {
const exit = await launch(argument, token)
expect(exit.code).toBe(1)
expect(exit.stdout).toBe('')
expect(exit.stderr).toContain(reason)
expect(exit.stderr.endsWith('\n')).toBe(true)
expect(exit.stderr).not.toMatch(/^\s+at /mu)
expect(exit.stderr).not.toContain('Node.js v')
}, 30_000)
+21 -16
View File
@@ -4,20 +4,25 @@ import { fileURLToPath } from 'node:url'
import { isSea } from 'node:sea'
import { runSshHelper } from './helper.ts'
const controller = new AbortController()
const stop = (): void => { controller.abort(new Error('SSH helper process terminated')) }
for (const signal of ['SIGTERM', 'SIGHUP', 'SIGINT'] as const) process.once(signal, stop)
try {
if (process.argv.length !== 2) throw new Error('SSH helper accepts no command arguments')
const packaged = isSea()
await runSshHelper({
input: process.stdin, output: process.stdout,
entryPath: packaged ? process.execPath : fileURLToPath(import.meta.url),
kind: packaged ? 'executable' : 'node-script', signal: controller.signal,
})
} catch (error) {
process.stderr.write(`dsh-ssh-sandbox: ${error instanceof Error ? error.message : String(error)}\n`)
process.exitCode = 127
} finally {
for (const signal of ['SIGTERM', 'SIGHUP', 'SIGINT'] as const) process.off(signal, stop)
/** Serve one helper request; every failure is reported on stderr with exit code 127. */
async function main(): Promise<void> {
const controller = new AbortController()
const stop = (): void => { controller.abort(new Error('SSH helper process terminated')) }
for (const signal of ['SIGTERM', 'SIGHUP', 'SIGINT'] as const) process.once(signal, stop)
try {
if (process.argv.length !== 2) throw new Error('SSH helper accepts no command arguments')
const packaged = isSea()
await runSshHelper({
input: process.stdin, output: process.stdout,
entryPath: packaged ? process.execPath : fileURLToPath(import.meta.url),
kind: packaged ? 'executable' : 'node-script', signal: controller.signal,
})
} catch (error) {
process.stderr.write(`dsh-ssh-sandbox: ${error instanceof Error ? error.message : String(error)}\n`)
process.exitCode = 127
} finally {
for (const signal of ['SIGTERM', 'SIGHUP', 'SIGINT'] as const) process.off(signal, stop)
}
}
void main()
+1 -1
View File
@@ -371,7 +371,7 @@ describe('gate graph validation', () => {
'rescope-vendor', 'publint', 'constraints', 'default-product-isolation', 'product-use', 'official-bundle-catalog', 'provider-data', 'package-dependencies', 'application-entrypoints',
'dsh-package-licenses', 'node-next-types',
'optional-dependency-imports', 'client-packages', 'client-ui-i18n', 'client-route-resolution', 'no-bare-dispatcher',
'plugin-record-callers', 'no-unknown-casts',
'plugin-record-callers', 'no-unknown-casts', 'no-top-level-await',
'cordis-config', 'runtime-closure',
])
})
+1
View File
@@ -393,6 +393,7 @@ function sharedHygieneGates(): Gate[] {
pnpmScript('no-bare-dispatcher', 'verify-no-bare-dispatcher', { label: 'proxy-aware dispatchers' }),
pnpmScript('plugin-record-callers', 'verify-plugin-record-callers', { label: 'plugin record callers' }),
pnpmScript('no-unknown-casts', 'verify-no-unknown-casts', { label: 'no new unknown casts' }),
pnpmScript('no-top-level-await', 'verify-no-top-level-await', { label: 'no module-scope await' }),
]
}
+220
View File
@@ -0,0 +1,220 @@
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import { binSources, excludedEntry, findTopLevelAwaits, scanRepository, sourceCorpus, SOURCE_GLOB } from './verify-no-top-level-await.ts'
const FILE = 'packages/group/pkg/src/entry.ts'
function forms(source: string, file = FILE): string[] {
return findTopLevelAwaits(file, source).map(violation => violation.what)
}
describe('module-scope await detection', () => {
it('rejects the process-entry shape the rule exists for', () => {
expect(forms(`
const config = parse(process.argv[2])
const { port } = await startServer(config)
process.stdout.write(JSON.stringify({ port }))
`)).toEqual(['await expression'])
})
it('reports the offending line number and text', () => {
expect(findTopLevelAwaits(FILE, 'const a = 1\n await ready()\n')).toEqual([
{ file: FILE, line: 2, what: 'await expression', text: 'await ready()' },
])
})
it.each([
['a top-level if', 'if (enabled) { await ready() }'],
['an else branch', 'if (enabled) {} else await ready()'],
['a bare block', '{ await ready() }'],
['a try block', 'try { await ready() } catch {}'],
['a catch block', 'try { start() } catch { await stop() }'],
['a finally block', 'try { start() } finally { await stop() }'],
['a loop body', 'for (const item of items) await handle(item)'],
['a while body', 'while (pending()) await next()'],
['a switch case', 'switch (mode) { case 1: await run() }'],
['a conditional expression', 'const value = ready ? await load() : undefined'],
['an exported initializer', 'export const value = await load()'],
['a class computed member name', 'class Probe { [await key()]() {} }'],
['an object computed method name', 'const probe = { [await key()]() {} }'],
])('rejects an await nested in %s', (_form, source) => {
expect(forms(source)).toEqual(['await expression'])
})
it('rejects a module-scope for await loop', () => {
expect(forms('for await (const chunk of stream) handle(chunk)')).toEqual(['for await loop'])
})
it('rejects a for await loop nested in a top-level block', () => {
expect(forms('if (stream) { for await (const chunk of stream) handle(chunk) }')).toEqual(['for await loop'])
})
it('rejects a module-scope await using declaration', () => {
expect(forms('await using handle = await open()')).toEqual(['await using declaration', 'await expression'])
})
it('rejects a module-scope await in a TSX module', () => {
expect(forms('const element = <div>{await label()}</div>', 'packages/group/pkg/src/view.tsx'))
.toEqual(['await expression'])
})
it.each([
['a function declaration', 'async function main() { await ready() }'],
['a function expression', 'const main = async function () { await ready() }'],
['an arrow block body', 'const main = async () => { await ready() }'],
['an arrow concise body', 'const main = async () => await ready()'],
['an object method', 'const probe = { async run() { await ready() } }'],
['a class method', 'class Probe { async run() { await ready() } }'],
['a function nested in a top-level block', 'if (enabled) { void (async () => { await ready() })() }'],
['a for await loop inside a function', 'async function drain(stream) { for await (const chunk of stream) handle(chunk) }'],
['an await using declaration inside a function', 'async function use() { await using handle = open() }'],
['a parameter initializer, which runs when the function is called', 'function probe(value = await ready()) {}'],
['a class static block, which runs synchronously at class definition', 'class Probe { static { await ready() } }'],
])('accepts an await inside %s', (_form, source) => {
expect(forms(source)).toEqual([])
})
it('accepts the word await in strings, templates, and comments', () => {
expect(forms([
"const quoted = 'await ready()'",
'const template = `var { connect } = await import(${JSON.stringify(url)})`',
'// await ready()',
'/* for await (const x of y) */',
].join('\n'))).toEqual([])
})
it('accepts a synchronous using declaration', () => {
expect(forms('using handle = open()')).toEqual([])
})
})
describe('package source corpus', () => {
const roots: string[] = []
afterEach(() => {
for (const directory of roots.splice(0)) rmSync(directory, { recursive: true, force: true })
})
function fixture(files: Record<string, string>): string {
const directory = mkdtempSync(join(tmpdir(), 'dsh-top-level-await-'))
roots.push(directory)
mkdirSync(join(directory, 'packages'), { recursive: true })
for (const [path, text] of Object.entries(files)) {
mkdirSync(dirname(join(directory, path)), { recursive: true })
writeFileSync(join(directory, path), text)
}
return directory
}
it('covers every package source directory with TypeScript or TSX files', () => {
const directory = fixture({
'packages/a/one/src/index.ts': 'export const one = 1\n',
'packages/a/one/src/view.tsx': 'export const view = <div />\n',
'packages/b/two/src/nested/deep.ts': 'await ready()\n',
'packages/b/docs-only/README.md': '# no sources\n',
'packages/b/types-only/src/env.d.ts': 'declare const value: number\n',
})
expect(sourceCorpus(directory)).toEqual([
'packages/a/one/src/index.ts',
'packages/a/one/src/view.tsx',
'packages/b/two/src/nested/deep.ts',
'packages/b/types-only/src/env.d.ts',
])
expect(scanRepository(directory)).toEqual({
checked: 4,
excluded: [],
violations: [
{ file: 'packages/b/two/src/nested/deep.ts', line: 1, what: 'await expression', text: 'await ready()' },
],
})
})
it('excludes a source that starts with a shebang line, and only that file', () => {
const directory = fixture({
'packages/a/tool/src/cli.ts': '#!/usr/bin/env node\nawait main()\n',
'packages/a/tool/src/library.ts': 'await load()\n',
'packages/a/tool/src/view.tsx': '',
})
expect(scanRepository(directory)).toEqual({
checked: 2,
excluded: [{ file: 'packages/a/tool/src/cli.ts', reason: 'shebang' }],
violations: [{ file: 'packages/a/tool/src/library.ts', line: 1, what: 'await expression', text: 'await load()' }],
})
})
it('excludes the sources of build outputs a manifest bin field names, in map and string forms', () => {
const directory = fixture({
'packages/a/tool/package.json': JSON.stringify({ name: 'tool', bin: { tool: './lib/cli/main.js' } }),
'packages/a/tool/src/cli/main.ts': 'await main()\n',
'packages/a/tool/src/main.ts': 'await main()\n',
'packages/b/view/package.json': JSON.stringify({ name: 'view', bin: 'lib/run.js' }),
'packages/b/view/src/run.tsx': 'await render()\n',
})
expect(scanRepository(directory)).toEqual({
checked: 1,
excluded: [
{ file: 'packages/a/tool/src/cli/main.ts', reason: 'bin' },
{ file: 'packages/b/view/src/run.tsx', reason: 'bin' },
],
violations: [{ file: 'packages/a/tool/src/main.ts', line: 1, what: 'await expression', text: 'await main()' }],
})
})
it('checks a source whose bin target is a committed file rather than a build output', () => {
const directory = fixture({
'packages/a/tool/package.json': JSON.stringify({ name: 'tool', bin: { tool: './bin.js' } }),
'packages/a/tool/src/bin.ts': 'await main()\n',
'packages/a/tool/src/view.tsx': '',
})
expect(scanRepository(directory).violations.map(violation => violation.file)).toEqual(['packages/a/tool/src/bin.ts'])
})
it('treats only a leading #! as a shebang', () => {
expect(excludedEntry(FILE, '// #!/usr/bin/env node\nawait main()\n', new Set())).toBeUndefined()
expect(excludedEntry(FILE, ' #!/usr/bin/env node\n', new Set())).toBeUndefined()
expect(excludedEntry(FILE, '#!/usr/bin/env node\n', new Set())).toBe('shebang')
})
it.each([
['an absent bin field', undefined, []],
['a lib target', 'lib/cli.js', ['packages/a/tool/src/cli.ts', 'packages/a/tool/src/cli.tsx']],
['a dotted lib target in a name map', { tool: './lib/nested/cli.js' }, ['packages/a/tool/src/nested/cli.ts', 'packages/a/tool/src/nested/cli.tsx']],
['a committed root file', { tool: './bin.js' }, []],
['a non-JavaScript lib file', 'lib/cli.sh', []],
['a non-string map value', { tool: 1 }, []],
])('maps %s to its source candidates', (_case, bin, expected) => {
expect(binSources('packages/a/tool', bin)).toEqual(expected)
})
it('refuses an empty corpus', () => {
expect(() => sourceCorpus(fixture({ 'packages/a/one/README.md': '# empty\n' }))).toThrow('empty corpus')
})
it('refuses a corpus that no longer holds TSX sources', () => {
const directory = fixture({ 'packages/a/one/src/index.ts': '', 'packages/a/one/src/view.tsx': '' })
expect(() => sourceCorpus(directory, 'packages/*/*/src/**/*.ts')).toThrow('no .tsx file')
})
it('refuses a glob narrowed to fewer package source directories', () => {
const directory = fixture({
'packages/a/one/src/view.tsx': '',
'packages/b/two/src/index.ts': '',
})
expect(() => sourceCorpus(directory, 'packages/a/*/src/**/*.{ts,tsx}')).toThrow('leaves out package sources: packages/b/two/src')
})
it('reads the shipped corpus through the documented glob', () => {
expect(SOURCE_GLOB).toBe('packages/*/*/src/**/*.{ts,tsx}')
expect(sourceCorpus().length).toBeGreaterThan(0)
})
it('passes on the current tree, excluding only the declared executables', () => {
const result = scanRepository()
expect(result.violations).toEqual([])
expect(result.excluded).toEqual([
{ file: 'packages/experimental/webworker-packer/src/bin.ts', reason: 'shebang' },
{ file: 'packages/test-support/llm-mock-server/src/bin.ts', reason: 'shebang' },
])
}, 60_000)
})
+267
View File
@@ -0,0 +1,267 @@
/**
* Verify that no importable package source module awaits at module scope.
*
* A module-scope `await` makes module evaluation asynchronous: every importer's
* evaluation then waits on it, a rejection surfaces as a module-evaluation
* failure instead of an error the module handles, and the module cannot be
* lowered to the synchronous CommonJS wrapper the browser-worker image loads.
* Package code does its asynchronous work inside a function it calls, such as an
* explicit `main()` in a process entry, and handles that promise's failure.
*
* Executables are not imported, so they are excluded by declaration, never by
* file name: a source file that starts with a `#!` line, and the source of a
* build output that a package manifest's `bin` field names. The repository build
* emits `src/<path>.ts` and `src/<path>.tsx` as `lib/<path>.js`; a `bin` target
* outside `lib/` is not a build output and excludes no source file.
*
* Discovery is syntax-aware, as `scripts/AGENTS.md` requires: the TypeScript
* parser finds `await` expressions, `for await` loops, and `await using`
* declarations whose nearest enclosing function body is the module itself,
* including those nested in top-level blocks, conditionals, loops, and
* class computed member names. Text inside strings, template literals, and
* comments is not code, and the parser does not report it.
*/
import { existsSync, globSync, readdirSync, readFileSync } from 'node:fs'
import { join, posix, resolve } from 'node:path'
import ts from 'typescript'
const GATE = 'verify-no-top-level-await'
const root = resolve(import.meta.dirname, '..')
/** The corpus glob for every TypeScript and TSX file under a workspace package's `src/`. */
export const SOURCE_GLOB = 'packages/*/*/src/**/*.{ts,tsx}'
/** Block-scope flag combination TypeScript records for an `await using` declaration list. */
const AWAIT_USING: number = ts.NodeFlags.AwaitUsing
/** Why a corpus file is not checked. */
export type ExcludedEntry = 'shebang' | 'bin'
/**
* Source files a package manifest's `bin` field runs.
* @param packageDirectory - repository-relative POSIX package directory.
* @param bin - the manifest's `bin` field: a path, a name-to-path map, or absent.
* @returns repository-relative candidate sources, `.ts` and `.tsx`, for each `lib/<path>.js` target.
*/
export function binSources(packageDirectory: string, bin: unknown): string[] {
const targets = typeof bin === 'string'
? [bin]
: typeof bin === 'object' && bin !== null ? Object.values(bin).filter(value => typeof value === 'string') : []
return targets.flatMap((target) => {
const match = /^lib\/(.+)\.js$/u.exec(posix.normalize(target))
if (match === null) return []
return ['ts', 'tsx'].map(extension => `${packageDirectory}/src/${match[1] as string}.${extension}`)
})
}
/**
* Read the `bin`-named sources of every package that owns a corpus file.
* @param repositoryRoot - absolute repository root.
* @param corpus - repository-relative corpus files.
* @returns repository-relative sources excluded as `bin` entries.
*/
function binSourcesOfCorpus(repositoryRoot: string, corpus: readonly string[]): Set<string> {
const sources = new Set<string>()
for (const directory of new Set(corpus.map(file => file.split('/').slice(0, 3).join('/')))) {
const manifest = join(repositoryRoot, directory, 'package.json')
if (!existsSync(manifest)) continue
const { bin } = JSON.parse(readFileSync(manifest, 'utf8')) as { bin?: unknown }
for (const source of binSources(directory, bin)) sources.add(source)
}
return sources
}
/**
* Classify one corpus file as an executable excluded from the check.
* @param file - repository-relative POSIX path.
* @param sourceText - the file's contents.
* @param bins - sources named by package `bin` fields.
* @returns the exclusion reason, or undefined for an importable module.
*/
export function excludedEntry(file: string, sourceText: string, bins: ReadonlySet<string>): ExcludedEntry | undefined {
if (sourceText.startsWith('#!')) return 'shebang'
if (bins.has(file)) return 'bin'
return undefined
}
/** One module-scope await position. */
export interface TopLevelAwaitViolation {
/** Repository-relative path, in POSIX separators. */
readonly file: string
/** One-based line number. */
readonly line: number
/** Which await form was found. */
readonly what: 'await expression' | 'for await loop' | 'await using declaration'
/** The offending source line, trimmed. */
readonly text: string
}
/**
* Whether a node starts a new function body, so an `await` below it belongs to
* that function rather than to the module.
* @param node - an ancestor of the await position.
* @returns true for every function-like node and class static blocks.
*/
function startsFunctionScope(node: ts.Node): node is ts.FunctionLikeDeclaration | ts.ClassStaticBlockDeclaration {
return ts.isFunctionDeclaration(node) || ts.isFunctionExpression(node) || ts.isArrowFunction(node)
|| ts.isMethodDeclaration(node) || ts.isConstructorDeclaration(node)
|| ts.isGetAccessorDeclaration(node) || ts.isSetAccessorDeclaration(node)
|| ts.isClassStaticBlockDeclaration(node)
}
/**
* Whether `child` lies in the part of `scope` that runs when the function is
* called: its parameters or body. A method's computed name and decorators run
* where the method is defined, so an await there still belongs to the module.
* @param scope - the function-like ancestor.
* @param child - the ancestor's direct child on the path to the await.
* @returns true when the await runs inside the function.
*/
function runsInsideScope(scope: ts.FunctionLikeDeclaration | ts.ClassStaticBlockDeclaration, child: ts.Node): boolean {
if (ts.isClassStaticBlockDeclaration(scope)) return true
return child === scope.body || scope.parameters.some(parameter => parameter === child)
}
/**
* Whether a node evaluates at module scope.
* @param node - an await-bearing node.
* @returns true when no enclosing function body contains it.
*/
function atModuleScope(node: ts.Node): boolean {
let child = node
for (let parent = node.parent; !ts.isSourceFile(parent); child = parent, parent = parent.parent) {
if (startsFunctionScope(parent) && runsInsideScope(parent, child)) return false
}
return true
}
/**
* Find every module-scope await in one source file.
* @param file - repository-relative path, used for the script kind and the report.
* @param sourceText - the file's contents.
* @returns one violation per await position, in source order.
*/
export function findTopLevelAwaits(file: string, sourceText: string): TopLevelAwaitViolation[] {
const posix = file.replaceAll('\\', '/')
// Every await form spells the keyword, so a file without it holds no violation and needs no parse.
if (!sourceText.includes('await')) return []
const kind = posix.endsWith('.tsx') ? ts.ScriptKind.TSX : ts.ScriptKind.TS
const source = ts.createSourceFile(posix, sourceText, ts.ScriptTarget.Latest, true, kind)
const lines = sourceText.split('\n')
const violations: TopLevelAwaitViolation[] = []
const record = (node: ts.Node, what: TopLevelAwaitViolation['what']): void => {
const line = source.getLineAndCharacterOfPosition(node.getStart(source)).line
violations.push({ file: posix, line: line + 1, what, text: (lines[line] ?? '').trim() })
}
const visit = (node: ts.Node): void => {
if (ts.isAwaitExpression(node) && atModuleScope(node)) record(node, 'await expression')
else if (ts.isForOfStatement(node) && node.awaitModifier !== undefined && atModuleScope(node)) {
record(node, 'for await loop')
} else if (ts.isVariableDeclarationList(node)
&& (node.flags & ts.NodeFlags.BlockScoped) === AWAIT_USING && atModuleScope(node)) {
record(node, 'await using declaration')
}
ts.forEachChild(node, visit)
}
ts.forEachChild(source, visit)
return violations
}
/**
* List the package source corpus and prove it covers every package source directory.
* @param repositoryRoot - absolute repository root.
* @param pattern - corpus glob; only tests pass another value, to prove a narrowed glob is refused.
* @returns repository-relative POSIX paths, sorted.
* @throws when the corpus is empty, contains no TSX file, or leaves out a package `src/` directory
* that holds TypeScript sources, because the gate would then pass by scanning less than it claims.
*/
export function sourceCorpus(repositoryRoot: string = root, pattern: string = SOURCE_GLOB): string[] {
const files = globSync(pattern, { cwd: repositoryRoot }).map(file => file.replaceAll('\\', '/')).sort()
if (files.length === 0) throw new Error(`${GATE}: scanned an empty corpus; ${pattern} no longer matches.`)
if (!files.some(file => file.endsWith('.tsx'))) {
throw new Error(`${GATE}: the corpus holds no .tsx file; ${pattern} no longer covers TSX sources.`)
}
const covered = new Set(files.map(file => file.split('/').slice(0, 3).join('/')))
const missing: string[] = []
for (const group of readdirSync(join(repositoryRoot, 'packages'), { withFileTypes: true })) {
if (!group.isDirectory()) continue
for (const pkg of readdirSync(join(repositoryRoot, 'packages', group.name), { withFileTypes: true })) {
const directory = `packages/${group.name}/${pkg.name}`
if (!pkg.isDirectory() || covered.has(directory)) continue
if (holdsTypeScript(join(repositoryRoot, directory, 'src'))) missing.push(`${directory}/src`)
}
}
if (missing.length > 0) {
throw new Error(`${GATE}: the corpus leaves out package sources: ${missing.join(', ')}.`)
}
return files
}
/**
* Whether a directory tree contains a TypeScript or TSX file, walked without the gate's glob.
* @param directory - absolute directory path, which may be absent.
* @returns true when at least one `.ts` or `.tsx` file lies below it.
*/
function holdsTypeScript(directory: string): boolean {
let entries
try {
entries = readdirSync(directory, { withFileTypes: true })
} catch (error) {
// A package without a src/ directory contributes nothing to the corpus.
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return false
throw error
}
return entries.some(entry => entry.isDirectory()
? holdsTypeScript(join(directory, entry.name))
: /\.tsx?$/u.test(entry.name))
}
/** What one repository scan checked, excluded, and found. */
export interface ScanResult {
/** Corpus files checked as importable modules. */
readonly checked: number
/** Corpus files excluded as executables, with the reason, in corpus order. */
readonly excluded: ReadonlyArray<{ readonly file: string; readonly reason: ExcludedEntry }>
/** Module-scope awaits in checked files, in corpus order. */
readonly violations: readonly TopLevelAwaitViolation[]
}
/**
* Scan the package source corpus, excluding executables by declaration.
* @param repositoryRoot - absolute repository root.
* @returns the checked count, the excluded executables, and every violation.
*/
export function scanRepository(repositoryRoot: string = root): ScanResult {
const corpus = sourceCorpus(repositoryRoot)
const bins = binSourcesOfCorpus(repositoryRoot, corpus)
const excluded: Array<{ file: string; reason: ExcludedEntry }> = []
const violations: TopLevelAwaitViolation[] = []
for (const file of corpus) {
const text = readFileSync(join(repositoryRoot, file), 'utf8')
const reason = excludedEntry(file, text, bins)
if (reason === undefined) violations.push(...findTopLevelAwaits(file, text))
else excluded.push({ file, reason })
}
return { checked: corpus.length - excluded.length, excluded, violations }
}
function main(): void {
const { checked, excluded, violations } = scanRepository()
const exclusions = excluded.map(entry => `${entry.file} (${entry.reason})`).join(', ')
if (violations.length === 0) {
console.log(`${GATE}: ${String(checked)} package source file(s) contain no module-scope await; `
+ `${String(excluded.length)} executable(s) excluded: ${exclusions || 'none'}.`)
return
}
console.error(`${GATE}: ${String(violations.length)} module-scope await(s) in package sources:\n`)
for (const violation of violations) {
console.error(` ${violation.file}:${String(violation.line)} ${violation.what}`)
console.error(` ${violation.text}`)
}
console.error('\nMove the asynchronous work into a function the module calls, such as an explicit `main()`,')
console.error('and handle that promise\'s failure where the module starts it.')
process.exitCode = 1
}
if (import.meta.filename === resolve(process.argv[1] ?? '')) main()