fix(test): prepare preview generations and preserve embedded stream timing

This commit is contained in:
Tianyi Cui
2026-09-16 13:46:05 +08:00
parent 8a46149577
commit 3bde1a56e3
21 changed files with 273 additions and 27 deletions
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md
2026-08-31-released-session-format-migrations.md: 6c236431a69a004d969b321f896041b791b2900a
2026-08-31-released-session-format-migrations.zh.md: 3c0e242770a5056c621e971befb023869926bf6f
2026-08-31-released-session-format-migrations.md: 8d7b8cbc06b77fbe3587593df65f454cf7752518
2026-08-31-released-session-format-migrations.zh.md: 0443740ff1b724cd4f27dd636cd3dff381824f5a
@@ -99,6 +99,8 @@ Current encoding is record based. The provider serializes about 1 MiB of plainte
The shipped `lib/worker.cjs` bundles its JavaScript workspace dependencies so each fresh verifier avoids resolving and compiling their runtime module graph. This is safe because the Worker communicates through plain request/result messages and shares no service or class identity with its host. The Host build applies the existing TypeScript and Typert transforms; the Client pass skips this Node-only package instead of replacing its worker with untransformed source. Native add-ons remain external. Verification, scheduler admission, termination, and durable publication still complete before writable open returns. The built-worker smoke copies the package manifest and worker into an isolated temporary package with ambient module paths removed, accepts a valid generation, and rejects an incorrect event count.
The [browser preview packer](../../../../packages/experimental/webworker-packer/README.md) prepares current successors for bundled Session data in Node. The browser host does not implement `node:worker_threads`, so this preparation preserves both immutable fixture generations and runtime JSONL’s Worker verification requirement.
Preparation forwards cancellation through source reads and observes it at the existing approximately 500 ms Decode yield boundary. Once `publish()` starts, encode, Worker verification, and publication do not receive caller cancellation and run to settlement; write open checks its caller signal again afterward. A published generation is never rolled back.
The Stage pipeline ends at one prepared current artifact. [Historical Session read preparation](2026-09-05-read-only-session-migration-preparation.md) defines how read open consumes that artifact immediately while write open performs encode, verification, and publication before returning append access.
@@ -99,6 +99,8 @@ Current encode 以单条 record 为单位。Provider 在主线程每个 slice
发布的 `lib/worker.cjs` 将 JavaScript workspace 依赖一起打包,使每个新 verifier 无需解析并编译它们的运行时模块图。Worker 只通过普通 request/result 消息通信,与 host 不共享 service 或 class identity,因此可以这样处理。Host build 应用现有 TypeScript 与 Typert 转换;Client pass 跳过这个 Node-only package,不会用未经转换的源代码覆盖 worker。Native add-on 保持 external。Verification、scheduler admission、termination 与 durable publication 仍在 writable open 返回前完成。Built-worker 冒烟测试把 package manifest 与 worker 复制到隔离的临时 package,移除环境中的模块搜索路径,接受有效 generation,并拒绝错误的 event count。
[浏览器 preview 打包器](../../../../packages/experimental/webworker-packer/README.zh.md)在 Node 中为随包 Session 数据准备当前代际后继。浏览器 host 未实现 `node:worker_threads`,因此打包阶段的准备同时保留不可变 fixture 代际与运行时 JSONL 的 Worker 校验要求。
Preparation 会把 cancellation 传给 source read,并在现有的约 500 ms Decode yield 边界观察它。`publish()` 一旦开始,encode、Worker verification 与 publication 不接收 caller cancellation,并运行到终态;write open 会在之后再次检查 caller signal。已经发布的 generation 绝不会回滚。
Stage pipeline 终止于一份 prepared current artifact。[历史 Session 只读迁移准备](2026-09-05-read-only-session-migration-preparation.zh.md)定义 read open 如何立即消费该 artifact,以及 write open 如何在返回 append 权限前完成 encode、verification 与 publication。
+5 -5
View File
@@ -27,7 +27,7 @@ import { chromium } from 'playwright'
import type { Browser } from 'playwright'
import { expect, it } from 'vitest'
import {
composeProfile, configTrees, indexWorkspacePackages, packVfsImage, packVfsOverlay,
composeProfile, configTrees, indexWorkspacePackages, packVfsImage, packPreviewFixture,
previewFixtures, WRAPPER_CONTRACT,
} from '@deepseek-ai/dsh-experimental-webworker-packer'
import {
@@ -118,9 +118,9 @@ function requirePreviewPages(): void {
* The base image, fixture manifest, and overlays to serve. `pnpm run build`
* emits the pages but only `build:preview` packs the image, so this lane packs
* a missing image rather than skipping the deployment it accepts. The example
* overlay pairs its committed Session generations with the generator-owned
* current projection cache. The worker therefore exercises historical reads
* without relying on a stale cache schema. Generated files land in a temp
* overlay retains its committed Session generations, prepares current successors
* through the Node catalog, and supplies the generator-owned current projection cache.
* Generated files land in a temp
* directory, never in `dist/`: the
* client-artifact digest record treats `dist/` as build-owned, so a test write
* there fails the record check for every later consumer.
@@ -161,7 +161,7 @@ function requireVfsAssets(): PreviewAssets {
const trees = fixture.id === 'vfs-example'
? [...fixture.trees, { mount: 'home/storages', directory: cacheDirectory }]
: fixture.trees
writeAsset(relativePath, packVfsOverlay(trees).image)
writeAsset(relativePath, packPreviewFixture(trees).image)
return {
id: fixture.id,
label: fixture.label,
+30 -1
View File
@@ -1,9 +1,11 @@
import { afterEach, describe, expect, it } from 'vitest'
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { parseSessionLog, prepareSessionSnapshotFixtureForComparison } from '@deepseek-ai/dsh-llm-replay'
import {
assertFixtureInventory,
parseSeedFixture,
recordedSessionFixturePath,
selectedSessionFixture,
} from './scaffold.ts'
@@ -14,6 +16,33 @@ afterEach(async () => {
for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true })
})
describe('Web seed stream timing', () => {
it.each([2, 3])('preserves embedded timestamps from format v%i and its current projection', async (version) => {
const fixture = await readFile(new URL(`../../../snapshots/session/skill-load/session.v${version}.jsonl`, import.meta.url), 'utf8')
const current = prepareSessionSnapshotFixtureForComparison(fixture)
const expected = parseSessionLog(current)
expect(parseSeedFixture(fixture).events).toEqual(expected)
expect(parseSeedFixture(current).events).toEqual(expected)
})
it.each(['session.jsonl', 'session.v1.jsonl'])('reconstructs positive stream intervals from %s', async (filename) => {
const fixture = await readFile(new URL(`../../../snapshots/session/text-turn/${filename}`, import.meta.url), 'utf8')
const messages = parseSeedFixture(fixture).events.filter(event => event.type === 'assistant/message')
expect(messages).toHaveLength(1)
const stream = messages[0]!.data.stream
expect(stream.length).toBeGreaterThan(1)
let previousEnd = -1
for (const record of stream) {
const start = 'time' in record ? record.time : record.time0
expect(start).toBeGreaterThan(previousEnd)
previousEnd = 'time' in record ? record.time : record.time0 + record.dt.reduce((total, delta) => total + delta, 0)
}
expect(previousEnd).toBeGreaterThan(0)
})
})
describe('Web snapshot generation filenames', () => {
it('selects the highest parent and child generations without counting retained inputs twice', async () => {
const root = await mkdtemp(join(tmpdir(), 'dsh-web-fixture-generations-'))
+8 -7
View File
@@ -1255,12 +1255,7 @@ export function realizeSeedFixture(scaffold: WebScaffold, fixtureText: string, i
}).join('\n')
}
/**
* Parse a committed web seed fixture through the replay reader.
* @param fixtureText - session JSONL fixture contents.
* @returns the current header line, parsed header, and logical events.
*/
/** Give a migrated fixture stream positive relative timing before its final wall-clock rebase. */
/** Give reconstructed V0/V1 chunk streams positive intervals before the final wall-clock rebase. */
function spreadMigratedSeedStream(
stream: SessionEvent<'assistant/message'>['data']['stream'],
): SessionEvent<'assistant/message'>['data']['stream'] {
@@ -1277,6 +1272,12 @@ function spreadMigratedSeedStream(
})
}
/**
* Parse a committed web seed fixture through the replay reader.
* Embedded streams retain their recorded timing; V0/V1 chunk streams receive positive relative intervals.
* @param fixtureText - session JSONL fixture contents.
* @returns the current header line, parsed header, and logical events.
*/
export function parseSeedFixture(fixtureText: string): {
headerLine: string
header: Record<string, unknown>
@@ -1291,7 +1292,7 @@ export function parseSeedFixture(fixtureText: string): {
const header = JSON.parse(headerLine) as Record<string, unknown>
if (header.type !== 'session') throw new Error('seed fixture must start with a session header')
const events = parseSessionLog(current).map((event) => {
if (sourceHeader.version === SESSION_FORMAT_VERSION) return event
if (sourceHeader.version !== 0 && sourceHeader.version !== 1) return event
if (event.type === 'assistant/message') {
return { ...event, data: { ...event.data, stream: spreadMigratedSeedStream(event.data.stream) } }
}
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/experimental/webworker-packer/README.md
README.md: 90723b7718393ed32005bd6146764044a8ec80b1
README.zh.md: 9d7df43bf2b49d7b4a0438fcdb96fdbe44cc326a
README.md: 65a428cb82ec490cb1ff64432c372f912cc529de
README.zh.md: 8d3335dafb448fa6d2cd27395bb92dd97a4fb6c0
@@ -35,7 +35,9 @@ The pack is a three-layer standard stack:
`repository.ts` owns the repo-shaped inputs (workspace scan of `vendor/`, `packages/`, `native/system/packages/`, and `apps/`; profile composition through the real CLI dump path); `pack.ts` owns none of them, so the same library packs a different tree by being called differently. The native scan makes the Landlock entry package an ordinary published-view dependency while its executable remains a Worker platform implementation. The CLI is `dsh-pack-vfs-image --out <file> [--profile web]`; `apps/web`'s `build:preview` runs it after the preview shell build.
The repository adapter also declares the preview-only fixture trees under `webworker-runtime/tests/fixtures/`. The CLI packs each named fixture into a separate deterministic overlay archive plus a browser-readable manifest. Overlay files bypass npm publish-view and module-reachability exclusions, so dot directories and example source files remain intact; their mounts are limited to `home/` and `workspace/`. `pack.ts` treats them as opaque bytes, and Session and Workspace interpretation stays in the runtime packages that own those formats.
The repository adapter also declares the preview-only fixture trees under `webworker-runtime/tests/fixtures/`. The CLI packs each named fixture into a separate deterministic overlay archive plus a browser-readable manifest. Overlay files bypass npm publish-view and module-reachability exclusions, so dot directories and example source files remain intact; their mounts are limited to `home/` and `workspace/`. `packVfsOverlay` treats all files as opaque bytes.
`packPreviewFixture` prepares the newest canonical raw Session generation in each fixture directory through the [Session format catalog](../../session/session-format-catalog/README.md) in Node. It strictly restores the source, encodes and strictly validates the current successor, and packs that successor beside unchanged source files. A malformed, unsupported, or future selected generation refuses the build without falling back. Temporary successors are removed after packing. Browser write-open then uses current data without invoking Node's migration-verification Worker. Projection caches remain owner-validated; an older generation cannot supply a current fold checkpoint.
-----
@@ -57,6 +59,7 @@ None; this package neither assembles nor sends a provider request.
- **Vendored package sources (`src/*.ts`) are excluded** — nothing resolves them at runtime; a future in-worker source-inspection feature would need a dedicated include rule.
- **The packer assumes built `lib/` artifacts are current**: it never compiles, so a stale workspace build packs stale bytes. Run the repository build first.
- **The CLI requires its original repository location**: `dsh-pack-vfs-image` finds the checkout relative to its own installed file and reads the source CLI, configuration trees, and preview fixtures there. An npm installation supports the parameterized library API; the CLI and repository helpers require a complete, built DeepSeek Harness checkout.
- **Preview Session inputs use raw JSONL** — `packPreviewFixture` rejects compressed and noncanonical Session generation filenames. Generic overlays remain byte-preserving.
<a id="dev-note"></a>
@@ -35,7 +35,9 @@ VFS 镜像打包器:把一份合成 profile 变成浏览器 worker 挂载为
`repository.ts` 拥有仓库形态输入(`vendor/`、`packages/`、`native/system/packages/` 与 `apps/` 的 workspace 扫描;经真 CLI dump 路径合成 profile);`pack.ts` 一概不拥有,同一库换参即可打另一棵树。Native 扫描使 Landlock 入口包成为普通发布视图依赖,其可执行文件仍由 Worker 平台实现。CLI 为 `dsh-pack-vfs-image --out <file> [--profile web]`;`apps/web` 的 `build:preview` 在预览壳构建后运行它。
仓库适配器还声明 `webworker-runtime/tests/fixtures/` 下仅用于 preview 的 fixture(测试前置数据) tree。CLI 会把每套具名 fixture 打成一份独立的确定性 overlay 归档,并写出浏览器可读的 manifest(元数据清单)。Overlay 文件绕过 npm 发布视图和模块可达性排除规则,因此点目录与示例源码会完整保留;其挂载位置仅限 `home/` 与 `workspace/`。`pack.ts` 把它们视为不透明字节;会话与 Workspace 的解释仍归拥有这些格式的运行时包。
仓库适配器还声明 `webworker-runtime/tests/fixtures/` 下仅用于 preview 的 fixture(测试前置数据) tree。CLI 会把每套具名 fixture 打成一份独立的确定性 overlay 归档,并写出浏览器可读的 manifest(元数据清单)。Overlay 文件绕过 npm 发布视图和模块可达性排除规则,因此点目录与示例源码会完整保留;其挂载位置仅限 `home/` 与 `workspace/`。`packVfsOverlay` 将所有文件视为不透明字节。
`packPreviewFixture` 在 Node 中通过 [Session 格式目录](../../session/session-format-catalog/README.zh.md)准备每个 fixture 目录里最新的规范原始 Session 代际。它严格恢复源文件、编码并严格校验当前后继文件,然后将后继文件与未改变的源文件一同打包。如果选中的代际格式错误、不受支持或属于未来版本,构建会拒绝,不回退。临时后继文件在打包后删除。浏览器以写模式打开时即可使用当前数据,无需调用 Node 的迁移校验 Worker。投影缓存仍由其所属包校验;较早代际不能提供当前折叠检查点。
-----
@@ -57,6 +59,7 @@ VFS 镜像打包器:把一份合成 profile 变成浏览器 worker 挂载为
- **vendored 包源码(`src/*.ts`)被排除**——运行时无人解析它们;未来若有 worker 内源码巡检功能需要专门的 include 规则。
- **打包器假定构建产物 `lib/` 是新鲜的**:它从不编译,工作区构建陈旧就会打包陈旧字节。先跑仓库构建。
- **CLI 必须位于原有仓库位置**:`dsh-pack-vfs-image` 相对于自身安装文件定位检出目录,并从中读取源码 CLI、配置树和预览 fixture。npm 安装支持参数化的库 API;CLI 和仓库辅助函数需要完整且已构建的 DeepSeek Harness 检出目录。
- **Preview Session 输入使用原始 JSONL** — `packPreviewFixture` 拒绝压缩文件与不规范的 Session 代际文件名。通用 overlay 仍逐字节保留输入。
<a id="dev-note"></a>
@@ -36,6 +36,8 @@
"@deepseek-ai/cordis-plugin-include": "workspace:^",
"@deepseek-ai/dsh-experimental-webworker-runtime": "workspace:^",
"@deepseek-ai/dsh-home-paths": "workspace:^",
"@deepseek-ai/dsh-session-format": "workspace:^",
"@deepseek-ai/dsh-session-format-catalog": "workspace:^",
"js-yaml": "^4.2.0",
"picomatch": "^4.0.4"
},
@@ -14,9 +14,9 @@ import {
PREVIEW_FIXTURE_MANIFEST_FILE, PREVIEW_FIXTURE_MANIFEST_VERSION,
type PreviewFixtureManifest,
} from '@deepseek-ai/dsh-experimental-webworker-runtime'
import { packVfsImage, packVfsOverlay } from './pack.ts'
import { packVfsImage } from './pack.ts'
import {
composeProfile, configTrees, describePack, indexWorkspacePackages, previewFixtures,
composeProfile, configTrees, describePack, indexWorkspacePackages, packPreviewFixture, previewFixtures,
} from './repository.ts'
/**
@@ -66,7 +66,7 @@ const fixtureDirectory = join(dirname(outputFile), 'fixtures')
mkdirSync(fixtureDirectory, { recursive: true })
const fixtureLines: string[] = []
const fixtures = fixtureDefinitions.map((fixture) => {
const packed = packVfsOverlay(fixture.trees)
const packed = packPreviewFixture(fixture.trees)
const file = `fixtures/${fixture.id}.tar.gz`
writeFileSync(join(dirname(outputFile), file), packed.image)
fixtureLines.push(` fixture overlay ${fixture.id} (${String(packed.image.byteLength)} B compressed)`)
@@ -11,6 +11,6 @@ export {
type ConfigTree, type ImageTree, type PackOptions, type PackOverlayResult, type PackResult,
} from './pack.ts'
export {
composeProfile, configTrees, describePack, indexWorkspacePackages, previewFixtures,
composeProfile, configTrees, describePack, indexWorkspacePackages, packPreviewFixture, previewFixtures,
type PreviewFixture,
} from './repository.ts'
@@ -0,0 +1,79 @@
/** Node-side Session preparation for browser Preview data overlays. */
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join, posix } from 'node:path'
import { isSessionFormatJsonObject, parseSessionFormatLogFilename, sessionFormatLogFilename } from '@deepseek-ai/dsh-session-format'
import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog'
import { packVfsOverlay, type ImageTree, type PackOverlayResult } from './pack.ts'
import type { ImageFiles } from './transform-image.ts'
interface PreviewSession {
readonly path: string
readonly directory: string
readonly id: string
readonly version: number
readonly bytes: Uint8Array
}
/**
* Pack Preview data with strictly validated current Session successors prepared in Node.
* Committed source bytes remain in the overlay; only the newest canonical raw generation
* of each Session is restored. Future, malformed, and unsupported selected data refuse the pack.
* @param trees - Ordered source trees under the Preview's home and workspace mounts.
* @returns Deterministic overlay containing unchanged inputs and final current successors.
*/
export function packPreviewFixture(trees: readonly ImageTree[]): PackOverlayResult {
const original = packVfsOverlay(trees)
const successors = new Map<string, string>()
for (const source of selectedSessions(original.files)) {
const text = new TextDecoder('utf-8', { fatal: true }).decode(source.bytes)
if (!text.endsWith('\n')) throw new Error(`preview fixture: ${source.path} has a torn physical tail`)
const [headerLine = '', ...rows] = text.slice(0, -1).split('\n')
const header: unknown = JSON.parse(headerLine)
if (!isSessionFormatJsonObject(header) || header['version'] !== source.version || header['id'] !== source.id) {
throw new Error(`preview fixture: ${source.path} disagrees with its Session header`)
}
const restore = sessionFormatCatalog.createRestore(header, { recovery: 'strict', validation: 'current' })
for (const row of rows) restore.decodeRow(JSON.parse(row))
const artifact = restore.finish()
if (source.version === sessionFormatCatalog.currentVersion) continue
const encodedHeader = sessionFormatCatalog.encodeCurrentHeader(artifact.header, artifact.inheritedEventCount)
const encodedEvents = artifact.events.map(event => sessionFormatCatalog.encodeCurrentEvent(event))
const verification = sessionFormatCatalog.createRestore(encodedHeader, { recovery: 'strict', validation: 'current' })
for (const event of encodedEvents) verification.decodeRow(event)
verification.finish()
successors.set(
posix.join(source.directory, sessionFormatLogFilename(sessionFormatCatalog.currentVersion)),
[encodedHeader, ...encodedEvents].map(row => JSON.stringify(row) + '\n').join(''),
)
}
if (successors.size === 0) return original
const directory = mkdtempSync(join(tmpdir(), 'dsh-preview-sessions-'))
try {
for (const [imagePath, content] of successors) {
const path = join(directory, imagePath.slice('home/sessions/'.length))
mkdirSync(dirname(path), { recursive: true })
writeFileSync(path, content, { flag: 'wx' })
}
return packVfsOverlay([...trees, { mount: 'home/sessions', directory }])
} finally {
rmSync(directory, { recursive: true, force: true })
}
}
function selectedSessions(files: ImageFiles): readonly PreviewSession[] {
const selected = new Map<string, PreviewSession>()
for (const [path, bytes] of Object.entries(files)) {
const match = /^home\/sessions\/[^/]+\/([^/]+)\/(session(?:\..*)?\.jsonl(?:\.zstd)?)$/u.exec(path)
if (match === null) continue
const version = parseSessionFormatLogFilename(match[2] as string)
if (version === undefined) throw new Error(`preview fixture: ${path} must name a canonical raw Session generation`)
const directory = posix.dirname(path)
const previous = selected.get(directory)
if (previous === undefined || previous.version < version) {
selected.set(directory, { path, directory, id: match[1] as string, version, bytes })
}
}
return [...selected.values()]
}
@@ -14,6 +14,8 @@ import { join, relative } from 'node:path'
import { DSH_HOME_ENV } from '@deepseek-ai/dsh-home-paths'
import type { ConfigTree, ImageTree, PackResult } from './pack.ts'
export { packPreviewFixture } from './preview.ts'
/**
* Repository directories scanned for workspace and vendored packages. The
* image only ever materializes runtime packages, which live here. The Landlock
@@ -76,6 +76,7 @@ it('loads both tarballs through plain Node and mounts their base image and overl
import * as packer from '@deepseek-ai/dsh-experimental-webworker-packer'
import * as runtime from '@deepseek-ai/dsh-experimental-webworker-runtime'
import * as client from '@deepseek-ai/dsh-experimental-webworker-runtime/client'
import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog'
for (const name of ['webworker-packer', 'webworker-runtime']) {
assert.equal(import.meta.resolve('@deepseek-ai/dsh-experimental-' + name),
new URL('./node_modules/@deepseek-ai/dsh-experimental-' + name + '/lib/index.js', import.meta.url).href)
@@ -93,6 +94,14 @@ it('loads both tarballs through plain Node and mounts their base image and overl
const overlay = packer.packVfsOverlay([{ mount: 'workspace', directory: fileURLToPath(new URL('./overlay', import.meta.url)) }])
runtime.loadVfsOverlay(await runtime.inflateImage(overlay.image, 'packed overlay'), '/dsh', vfs)
assert.equal(vfs.readFileSync('/dsh/workspace/hello.txt', 'utf8'), 'packed worker pair\\n')
mkdirSync('home/sessions/project/preview', { recursive: true })
const historical = JSON.stringify({ type: 'session', version: 3, id: 'preview', createdAt: 1, isSeeded: false, delegationDepth: 0 }) + '\\n'
writeFileSync('home/sessions/project/preview/session.v3.jsonl', historical)
const prepared = packer.packPreviewFixture([{ mount: 'home', directory: fileURLToPath(new URL('./home', import.meta.url)) }])
runtime.loadVfsOverlay(await runtime.inflateImage(prepared.image, 'prepared preview'), '/dsh', vfs)
assert.equal(vfs.readFileSync('/dsh/home/sessions/project/preview/session.v3.jsonl', 'utf8'), historical)
const current = JSON.parse(vfs.readFileSync('/dsh/home/sessions/project/preview/session.v' + sessionFormatCatalog.currentVersion + '.jsonl', 'utf8'))
assert.equal(current.version, sessionFormatCatalog.currentVersion)
console.log('packed image and overlay mounted')
`
await writeFile(join(root, 'consumer.mjs'), script)
@@ -0,0 +1,102 @@
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { afterEach, describe, expect, it } from 'vitest'
import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog'
import { sessionFormatLogFilename } from '@deepseek-ai/dsh-session-format'
import { packPreviewFixture } from '../src/preview.ts'
import { packVfsOverlay } from '../src/pack.ts'
import { previewFixtures } from '../src/repository.ts'
const repoRoot = fileURLToPath(new URL('../../../../', import.meta.url))
const roots: string[] = []
const currentVersion = sessionFormatCatalog.currentVersion
const text = (bytes: Uint8Array | undefined): string => new TextDecoder().decode(bytes)
afterEach(() => {
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true })
})
function source() {
const root = mkdtempSync(join(tmpdir(), 'dsh-preview-source-'))
roots.push(root)
const session = join(root, 'sessions', 'project', 'example')
mkdirSync(session, { recursive: true })
return {
trees: [{ mount: 'home', directory: root }],
put(version: number, body = '', overrides: Record<string, unknown> = {}) {
const path = join(session, sessionFormatLogFilename(version))
const header = { type: 'session', version, id: 'example', createdAt: 1, isSeeded: false, delegationDepth: 0, ...overrides }
writeFileSync(path, JSON.stringify(header) + '\n' + body)
return path
},
}
}
function restore(bytes: Uint8Array | undefined) {
const [header, ...rows] = text(bytes).trimEnd().split('\n').map(row => JSON.parse(row) as unknown)
const reader = sessionFormatCatalog.createRestore(header, { recovery: 'strict', validation: 'current' })
for (const row of rows) reader.decodeRow(row)
return reader.finish()
}
describe('Node preparation of Preview Session data', () => {
it('packs strictly migrated current successors beside every unchanged example file', () => {
const fixture = previewFixtures(repoRoot)[0]!
const original = packVfsOverlay(fixture.trees)
const projected = packPreviewFixture(fixture.trees)
const currentFiles = Object.keys(projected.files).filter(path => path.endsWith('/' + sessionFormatLogFilename(currentVersion)))
expect(currentFiles).toHaveLength(3)
for (const [path, bytes] of Object.entries(original.files)) expect(projected.files[path], path).toEqual(bytes)
for (const path of currentFiles) {
const sourcePath = path.replace(/\/session\.v\d+\.jsonl$/u, '/session.v3.jsonl')
expect(restore(projected.files[path])).toEqual(restore(original.files[sourcePath]))
expect(JSON.parse(text(projected.files[path]).split('\n')[0]!)).toMatchObject({ version: currentVersion })
}
expect(packPreviewFixture(fixture.trees).image).toEqual(projected.image)
expect(packVfsOverlay(fixture.trees).files).toEqual(original.files)
})
it('uses the highest generation and leaves an already-current image byte-identical', () => {
const fixture = source()
const older = fixture.put(3, '{invalid historical data}\n')
fixture.put(currentVersion)
const original = packVfsOverlay(fixture.trees)
expect(packPreviewFixture(fixture.trees).image).toEqual(original.image)
expect(text(original.files['home/sessions/project/example/session.v3.jsonl'])).toBe(readFileSync(older, 'utf8'))
})
it('keeps an overlay without Sessions opaque', () => {
const fixture = source()
expect(packPreviewFixture(fixture.trees)).toEqual(packVfsOverlay(fixture.trees))
})
it.each(['future', 'corrupt', 'unknown-required'] as const)('refuses selected %s data without falling back or changing inputs', (mode) => {
const fixture = source()
fixture.put(3)
const path = fixture.put(mode === 'future' ? currentVersion + 1 : currentVersion,
mode === 'corrupt' ? '{invalid JSON}\n'
: mode === 'unknown-required' ? JSON.stringify({ type: 'external/required', seq: 0, time: 1, data: null }) + '\n' : '')
const before = readFileSync(path)
expect(() => packPreviewFixture(fixture.trees)).toThrow()
expect(readFileSync(path)).toEqual(before)
})
it.each([{ version: 2 }, { id: 'other' }])('refuses selected metadata that disagrees with its location: %j', (overrides) => {
const fixture = source()
fixture.put(3, '', overrides)
expect(() => packPreviewFixture(fixture.trees)).toThrow('disagrees with its Session header')
})
it('refuses a torn physical tail and compressed Session generations', () => {
const fixture = source()
const path = fixture.put(3)
const bytes = readFileSync(path)
writeFileSync(path, bytes.subarray(0, -1))
expect(() => packPreviewFixture(fixture.trees)).toThrow('torn physical tail')
writeFileSync(path, bytes)
writeFileSync(join(dirname(path), sessionFormatLogFilename(currentVersion) + '.zstd'), 'compressed data')
expect(() => packPreviewFixture(fixture.trees)).toThrow('canonical raw Session generation')
})
})
@@ -19,6 +19,12 @@
},
{
"path": "../../util/home-paths"
},
{
"path": "../../session/session-format"
},
{
"path": "../../session/session-format-catalog"
}
]
}
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/experimental/webworker-runtime/README.md
README.md: 724e47a004fac83fe0f19a588faf0c227d843232
README.zh.md: 4e88b3cf10f98a423268c60b69269d2550d2c1f9
README.md: de423c3606f1fd24582b6db1d3ffd011bfd4e632
README.zh.md: ca5ca793e7c56b1b1fccb65b356f3c98c03cac2b
@@ -32,7 +32,7 @@ Three build artifacts and one source-owned process layer:
- **`src/shell/` (the worker's own process layer)** — a browser worker cannot fork, so `node:child_process` is not a stub but an implementation: `spawn` starts the command in its own Web Worker — this same bundle, told by its first frame to be a shell process — and reports it through the `ChildProcess` surface the subprocess service consumes. The command runs off the host's thread, `SIGKILL` terminates it whatever it is doing, and it reaches the VFS only by message (the host serves those frames). Worker platform executables preserve native-package protocols such as Landlock without replacing their JavaScript packages or coupling their implementations to `node:child_process`; ordinary commands use the package's evaluator and coreutils command table. The grammar is `@yarnpkg/parsers`' `parseShell`, while `execSync`/`fork` still refuse because they need a real process.
- **`lib/client.js` (page half)** — startup has two independent stages. `chooseWorkerHostSource({ image?, fixtureManifest? })` optionally owns the boot barrier and fixture manifest: without `preview-fixture` it waits at the source chooser, while a valid query selects directly; either path returns ordered overlays. `connectWorkerHost(worker, { image?, overlays? })` remains the public base-runtime connector; callers that skip the chooser get an empty overlay list. `apps/web` invokes both and supplies its statically bundled Worker. The opening `init` frame carries the base and ordered overlay URLs, the boot payload delivers the structured index-injection table, and `applyIndexInjections` executes it before the shell entry runs. Script preload rows are advisory and skipped because `/plugins` resources resolve only through the tunnel; `loadBundle` fetches each combo on first demand, embeds its tunnel-only source map as a Base64 data URL, and executes the script as a Blob. The tunnel also exposes fetch-shaped transport, the independent file-upload carrier, and the API client. Request frames preserve Blob bodies through structured clone and transfer `ReadableStream<Uint8Array>` ownership. The Host Worker streams both forms into the route, so neither browser thread creates a complete byte array for a generic-file upload.
Acceptance lives in `apps/web/tests/preview-boot.e2e.ts`, which serves the real built pages and drives the pre-boot chooser plus Worker activation in headless Chromium. The empty selection exercises first-run startup. The `vfs-example` overlay supplies ordinary workspace files and plaintext persistence artifacts for cold Workspace/Session discovery, tool presentation, subagent navigation, and history paging without a model request. The fixture generator owns current-generation logs and the projection cache; committed predecessor logs remain byte-identical alongside them. The chooser reserves WebFS as a separate user-authorized source; that provider does not read the built-in fixture.
Acceptance lives in `apps/web/tests/preview-boot.e2e.ts`, which serves the real built pages and drives the pre-boot chooser plus Worker activation in headless Chromium. The empty selection exercises first-run startup. The `vfs-example` overlay supplies ordinary workspace files and plaintext persistence artifacts for cold Workspace/Session discovery, tool presentation, subagent navigation, and history paging without a model request. The [packer](../webworker-packer/README.md) prepares current-generation successors from committed historical logs in Node; those source logs remain byte-identical in the overlay. The fixture generator owns the acceptance test's current projection cache. The chooser reserves WebFS as a separate user-authorized source; that provider does not read the built-in fixture.
The [built-bundle import sweep](tests/compile/transform-corpus-check.ts) checks bare Node imports after the library build. Its Dockkit exception accepts Node’s unknown-`.css`-extension refusal for any stylesheet rather than one pinned path; another extension, error code, or message, and an unexpectedly successful exempt import, remain errors. See the [stylesheet exemption decision](../../../.agents/notes/implemented/bug-fix/2026-09-10-built-bundle-css-exemption.md).
@@ -32,7 +32,7 @@ kind: "package-library"
- **`src/shell/`(worker 自己的进程层)**——浏览器 worker 无法 fork,所以 `node:child_process` 不是 stub 而是实现:`spawn` 把命令放进它自己的 Web Worker——就是这同一个束,由首帧告诉它「你是 shell 进程」——并以 subprocess 服务消费的 `ChildProcess` 面报告结果。命令不占宿主线程,`SIGKILL` 不管它在干什么都能终止它,而它只能靠消息触达 VFS(由宿主应答这些帧)。Worker 平台 executable 在不替换 JavaScript 包、也不把具体实现耦合进 `node:child_process` 的情况下保持 Landlock 等 native 包协议;普通命令使用本包的求值器与 coreutils 命令表。语法来自 `@yarnpkg/parsers` 的 `parseShell`,而 `execSync`/`fork` 依然拒绝,因为它们需要真进程。
- **`lib/client.js`(页面半)**——启动分为相互独立的两段。`chooseWorkerHostSource({ image?, fixtureManifest? })` 可选地拥有 boot barrier 与 fixture manifest:没有 `preview-fixture` 时停在来源选择面板,合法 query 则直接选择;两条路径都返回按序排列的 overlays。`connectWorkerHost(worker, { image?, overlays? })` 仍是公开的基础运行态连接器;调用方跳过选择器时 overlay 列表为空。`apps/web` 调用这两段并提供静态打包的 Worker。开局 `init` 帧携带基础镜像与按序排列的 overlay URL,boot 载荷送达结构化 index 注入表,`applyIndexInjections` 在壳入口运行前逐行执行。脚本 preload 行只是提示,因此会被跳过:`/plugins` 资源只能经 tunnel 解析,`loadBundle` 会在首次需要时获取 combo、把仅 tunnel 可达的 sourcemap 内嵌为 Base64 data URL,再以 Blob 执行脚本。Tunnel 还暴露 fetch 形式的传输、独立文件上传载体和 API 客户端。请求帧通过结构化克隆保留 Blob 请求体,并转移 `ReadableStream<Uint8Array>` 的所有权。Host Worker 将两种请求体都逐块送入路由,因此通用文件上传不会在任何浏览器线程创建完整字节数组。
验收在 `apps/web/tests/preview-boot.e2e.ts`:静态服务真实构建页面,在 headless Chromium 里驱动 pre-boot 选择面板与 Worker 激活。空白选择验证首次启动;`vfs-example` overlay 提供普通 workspace 文件与明文 persistence 产物,无需模型请求即可验证 Workspace/Session 冷发现、工具呈现、subagent 导航和历史分页。fixture 生成器负责当前代日志与投影缓存;已提交的前代日志逐字节保持不变,与它们并存。选择面板为 WebFS 保留独立的用户授权来源;该 provider 不读取内置 fixture。
验收在 `apps/web/tests/preview-boot.e2e.ts`:静态服务真实构建页面,在 headless Chromium 里驱动 pre-boot 选择面板与 Worker 激活。空白选择验证首次启动;`vfs-example` overlay 提供普通 workspace 文件与明文 persistence 产物,无需模型请求即可验证 Workspace/Session 冷发现、工具呈现、subagent 导航和历史分页。[打包器](../webworker-packer/README.zh.md)在 Node 中根据已提交的历史日志准备当前代后继文件;这些源日志在 overlay 中逐字节保持不变。fixture 生成器负责验收测试的当前投影缓存。选择面板为 WebFS 保留独立的用户授权来源;该 provider 不读取内置 fixture。
[已构建 bundle 导入检查](tests/compile/transform-corpus-check.ts)在库构建后检查裸 Node 导入。Dockkit 例外接受 Node 针对任意样式表报告的未知 `.css` 扩展名错误,而不限定某一固定路径;其他扩展名、错误码或消息,以及意外成功的豁免导入,仍然报错。参见 [样式表豁免决策](../../../.agents/notes/implemented/bug-fix/2026-09-10-built-bundle-css-exemption.zh.md)。
+6
View File
@@ -6495,6 +6495,12 @@ importers:
'@deepseek-ai/dsh-home-paths':
specifier: workspace:^
version: link:../../util/home-paths
'@deepseek-ai/dsh-session-format':
specifier: workspace:^
version: link:../../session/session-format
'@deepseek-ai/dsh-session-format-catalog':
specifier: workspace:^
version: link:../../session/session-format-catalog
js-yaml:
specifier: ^4.2.0
version: 4.3.1