mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-11-09 20:07:43 +08:00
fix(test): prepare preview generations and preserve embedded stream timing
This commit is contained in:
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md
|
||||
2026-08-31-released-session-format-migrations.md: 6c236431a69a004d969b321f896041b791b2900a
|
||||
2026-08-31-released-session-format-migrations.zh.md: 3c0e242770a5056c621e971befb023869926bf6f
|
||||
2026-08-31-released-session-format-migrations.md: 8d7b8cbc06b77fbe3587593df65f454cf7752518
|
||||
2026-08-31-released-session-format-migrations.zh.md: 0443740ff1b724cd4f27dd636cd3dff381824f5a
|
||||
|
||||
+2
@@ -99,6 +99,8 @@ Current encoding is record based. The provider serializes about 1 MiB of plainte
|
||||
|
||||
The shipped `lib/worker.cjs` bundles its JavaScript workspace dependencies so each fresh verifier avoids resolving and compiling their runtime module graph. This is safe because the Worker communicates through plain request/result messages and shares no service or class identity with its host. The Host build applies the existing TypeScript and Typert transforms; the Client pass skips this Node-only package instead of replacing its worker with untransformed source. Native add-ons remain external. Verification, scheduler admission, termination, and durable publication still complete before writable open returns. The built-worker smoke copies the package manifest and worker into an isolated temporary package with ambient module paths removed, accepts a valid generation, and rejects an incorrect event count.
|
||||
|
||||
The [browser preview packer](../../../../packages/experimental/webworker-packer/README.md) prepares current successors for bundled Session data in Node. The browser host does not implement `node:worker_threads`, so this preparation preserves both immutable fixture generations and runtime JSONL’s Worker verification requirement.
|
||||
|
||||
Preparation forwards cancellation through source reads and observes it at the existing approximately 500 ms Decode yield boundary. Once `publish()` starts, encode, Worker verification, and publication do not receive caller cancellation and run to settlement; write open checks its caller signal again afterward. A published generation is never rolled back.
|
||||
|
||||
The Stage pipeline ends at one prepared current artifact. [Historical Session read preparation](2026-09-05-read-only-session-migration-preparation.md) defines how read open consumes that artifact immediately while write open performs encode, verification, and publication before returning append access.
|
||||
|
||||
+2
@@ -99,6 +99,8 @@ Current encode 以单条 record 为单位。Provider 在主线程每个 slice
|
||||
|
||||
发布的 `lib/worker.cjs` 将 JavaScript workspace 依赖一起打包,使每个新 verifier 无需解析并编译它们的运行时模块图。Worker 只通过普通 request/result 消息通信,与 host 不共享 service 或 class identity,因此可以这样处理。Host build 应用现有 TypeScript 与 Typert 转换;Client pass 跳过这个 Node-only package,不会用未经转换的源代码覆盖 worker。Native add-on 保持 external。Verification、scheduler admission、termination 与 durable publication 仍在 writable open 返回前完成。Built-worker 冒烟测试把 package manifest 与 worker 复制到隔离的临时 package,移除环境中的模块搜索路径,接受有效 generation,并拒绝错误的 event count。
|
||||
|
||||
[浏览器 preview 打包器](../../../../packages/experimental/webworker-packer/README.zh.md)在 Node 中为随包 Session 数据准备当前代际后继。浏览器 host 未实现 `node:worker_threads`,因此打包阶段的准备同时保留不可变 fixture 代际与运行时 JSONL 的 Worker 校验要求。
|
||||
|
||||
Preparation 会把 cancellation 传给 source read,并在现有的约 500 ms Decode yield 边界观察它。`publish()` 一旦开始,encode、Worker verification 与 publication 不接收 caller cancellation,并运行到终态;write open 会在之后再次检查 caller signal。已经发布的 generation 绝不会回滚。
|
||||
|
||||
Stage pipeline 终止于一份 prepared current artifact。[历史 Session 只读迁移准备](2026-09-05-read-only-session-migration-preparation.zh.md)定义 read open 如何立即消费该 artifact,以及 write open 如何在返回 append 权限前完成 encode、verification 与 publication。
|
||||
|
||||
@@ -27,7 +27,7 @@ import { chromium } from 'playwright'
|
||||
import type { Browser } from 'playwright'
|
||||
import { expect, it } from 'vitest'
|
||||
import {
|
||||
composeProfile, configTrees, indexWorkspacePackages, packVfsImage, packVfsOverlay,
|
||||
composeProfile, configTrees, indexWorkspacePackages, packVfsImage, packPreviewFixture,
|
||||
previewFixtures, WRAPPER_CONTRACT,
|
||||
} from '@deepseek-ai/dsh-experimental-webworker-packer'
|
||||
import {
|
||||
@@ -118,9 +118,9 @@ function requirePreviewPages(): void {
|
||||
* The base image, fixture manifest, and overlays to serve. `pnpm run build`
|
||||
* emits the pages but only `build:preview` packs the image, so this lane packs
|
||||
* a missing image rather than skipping the deployment it accepts. The example
|
||||
* overlay pairs its committed Session generations with the generator-owned
|
||||
* current projection cache. The worker therefore exercises historical reads
|
||||
* without relying on a stale cache schema. Generated files land in a temp
|
||||
* overlay retains its committed Session generations, prepares current successors
|
||||
* through the Node catalog, and supplies the generator-owned current projection cache.
|
||||
* Generated files land in a temp
|
||||
* directory, never in `dist/`: the
|
||||
* client-artifact digest record treats `dist/` as build-owned, so a test write
|
||||
* there fails the record check for every later consumer.
|
||||
@@ -161,7 +161,7 @@ function requireVfsAssets(): PreviewAssets {
|
||||
const trees = fixture.id === 'vfs-example'
|
||||
? [...fixture.trees, { mount: 'home/storages', directory: cacheDirectory }]
|
||||
: fixture.trees
|
||||
writeAsset(relativePath, packVfsOverlay(trees).image)
|
||||
writeAsset(relativePath, packPreviewFixture(trees).image)
|
||||
return {
|
||||
id: fixture.id,
|
||||
label: fixture.label,
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { parseSessionLog, prepareSessionSnapshotFixtureForComparison } from '@deepseek-ai/dsh-llm-replay'
|
||||
import {
|
||||
assertFixtureInventory,
|
||||
parseSeedFixture,
|
||||
recordedSessionFixturePath,
|
||||
selectedSessionFixture,
|
||||
} from './scaffold.ts'
|
||||
@@ -14,6 +16,33 @@ afterEach(async () => {
|
||||
for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
describe('Web seed stream timing', () => {
|
||||
it.each([2, 3])('preserves embedded timestamps from format v%i and its current projection', async (version) => {
|
||||
const fixture = await readFile(new URL(`../../../snapshots/session/skill-load/session.v${version}.jsonl`, import.meta.url), 'utf8')
|
||||
const current = prepareSessionSnapshotFixtureForComparison(fixture)
|
||||
const expected = parseSessionLog(current)
|
||||
|
||||
expect(parseSeedFixture(fixture).events).toEqual(expected)
|
||||
expect(parseSeedFixture(current).events).toEqual(expected)
|
||||
})
|
||||
|
||||
it.each(['session.jsonl', 'session.v1.jsonl'])('reconstructs positive stream intervals from %s', async (filename) => {
|
||||
const fixture = await readFile(new URL(`../../../snapshots/session/text-turn/${filename}`, import.meta.url), 'utf8')
|
||||
const messages = parseSeedFixture(fixture).events.filter(event => event.type === 'assistant/message')
|
||||
expect(messages).toHaveLength(1)
|
||||
const stream = messages[0]!.data.stream
|
||||
expect(stream.length).toBeGreaterThan(1)
|
||||
|
||||
let previousEnd = -1
|
||||
for (const record of stream) {
|
||||
const start = 'time' in record ? record.time : record.time0
|
||||
expect(start).toBeGreaterThan(previousEnd)
|
||||
previousEnd = 'time' in record ? record.time : record.time0 + record.dt.reduce((total, delta) => total + delta, 0)
|
||||
}
|
||||
expect(previousEnd).toBeGreaterThan(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('Web snapshot generation filenames', () => {
|
||||
it('selects the highest parent and child generations without counting retained inputs twice', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-web-fixture-generations-'))
|
||||
|
||||
@@ -1255,12 +1255,7 @@ export function realizeSeedFixture(scaffold: WebScaffold, fixtureText: string, i
|
||||
}).join('\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a committed web seed fixture through the replay reader.
|
||||
* @param fixtureText - session JSONL fixture contents.
|
||||
* @returns the current header line, parsed header, and logical events.
|
||||
*/
|
||||
/** Give a migrated fixture stream positive relative timing before its final wall-clock rebase. */
|
||||
/** Give reconstructed V0/V1 chunk streams positive intervals before the final wall-clock rebase. */
|
||||
function spreadMigratedSeedStream(
|
||||
stream: SessionEvent<'assistant/message'>['data']['stream'],
|
||||
): SessionEvent<'assistant/message'>['data']['stream'] {
|
||||
@@ -1277,6 +1272,12 @@ function spreadMigratedSeedStream(
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a committed web seed fixture through the replay reader.
|
||||
* Embedded streams retain their recorded timing; V0/V1 chunk streams receive positive relative intervals.
|
||||
* @param fixtureText - session JSONL fixture contents.
|
||||
* @returns the current header line, parsed header, and logical events.
|
||||
*/
|
||||
export function parseSeedFixture(fixtureText: string): {
|
||||
headerLine: string
|
||||
header: Record<string, unknown>
|
||||
@@ -1291,7 +1292,7 @@ export function parseSeedFixture(fixtureText: string): {
|
||||
const header = JSON.parse(headerLine) as Record<string, unknown>
|
||||
if (header.type !== 'session') throw new Error('seed fixture must start with a session header')
|
||||
const events = parseSessionLog(current).map((event) => {
|
||||
if (sourceHeader.version === SESSION_FORMAT_VERSION) return event
|
||||
if (sourceHeader.version !== 0 && sourceHeader.version !== 1) return event
|
||||
if (event.type === 'assistant/message') {
|
||||
return { ...event, data: { ...event.data, stream: spreadMigratedSeedStream(event.data.stream) } }
|
||||
}
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/experimental/webworker-packer/README.md
|
||||
README.md: 90723b7718393ed32005bd6146764044a8ec80b1
|
||||
README.zh.md: 9d7df43bf2b49d7b4a0438fcdb96fdbe44cc326a
|
||||
README.md: 65a428cb82ec490cb1ff64432c372f912cc529de
|
||||
README.zh.md: 8d3335dafb448fa6d2cd27395bb92dd97a4fb6c0
|
||||
|
||||
@@ -35,7 +35,9 @@ The pack is a three-layer standard stack:
|
||||
|
||||
`repository.ts` owns the repo-shaped inputs (workspace scan of `vendor/`, `packages/`, `native/system/packages/`, and `apps/`; profile composition through the real CLI dump path); `pack.ts` owns none of them, so the same library packs a different tree by being called differently. The native scan makes the Landlock entry package an ordinary published-view dependency while its executable remains a Worker platform implementation. The CLI is `dsh-pack-vfs-image --out <file> [--profile web]`; `apps/web`'s `build:preview` runs it after the preview shell build.
|
||||
|
||||
The repository adapter also declares the preview-only fixture trees under `webworker-runtime/tests/fixtures/`. The CLI packs each named fixture into a separate deterministic overlay archive plus a browser-readable manifest. Overlay files bypass npm publish-view and module-reachability exclusions, so dot directories and example source files remain intact; their mounts are limited to `home/` and `workspace/`. `pack.ts` treats them as opaque bytes, and Session and Workspace interpretation stays in the runtime packages that own those formats.
|
||||
The repository adapter also declares the preview-only fixture trees under `webworker-runtime/tests/fixtures/`. The CLI packs each named fixture into a separate deterministic overlay archive plus a browser-readable manifest. Overlay files bypass npm publish-view and module-reachability exclusions, so dot directories and example source files remain intact; their mounts are limited to `home/` and `workspace/`. `packVfsOverlay` treats all files as opaque bytes.
|
||||
|
||||
`packPreviewFixture` prepares the newest canonical raw Session generation in each fixture directory through the [Session format catalog](../../session/session-format-catalog/README.md) in Node. It strictly restores the source, encodes and strictly validates the current successor, and packs that successor beside unchanged source files. A malformed, unsupported, or future selected generation refuses the build without falling back. Temporary successors are removed after packing. Browser write-open then uses current data without invoking Node's migration-verification Worker. Projection caches remain owner-validated; an older generation cannot supply a current fold checkpoint.
|
||||
|
||||
-----
|
||||
|
||||
@@ -57,6 +59,7 @@ None; this package neither assembles nor sends a provider request.
|
||||
- **Vendored package sources (`src/*.ts`) are excluded** — nothing resolves them at runtime; a future in-worker source-inspection feature would need a dedicated include rule.
|
||||
- **The packer assumes built `lib/` artifacts are current**: it never compiles, so a stale workspace build packs stale bytes. Run the repository build first.
|
||||
- **The CLI requires its original repository location**: `dsh-pack-vfs-image` finds the checkout relative to its own installed file and reads the source CLI, configuration trees, and preview fixtures there. An npm installation supports the parameterized library API; the CLI and repository helpers require a complete, built DeepSeek Harness checkout.
|
||||
- **Preview Session inputs use raw JSONL** — `packPreviewFixture` rejects compressed and noncanonical Session generation filenames. Generic overlays remain byte-preserving.
|
||||
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -35,7 +35,9 @@ VFS 镜像打包器:把一份合成 profile 变成浏览器 worker 挂载为
|
||||
|
||||
`repository.ts` 拥有仓库形态输入(`vendor/`、`packages/`、`native/system/packages/` 与 `apps/` 的 workspace 扫描;经真 CLI dump 路径合成 profile);`pack.ts` 一概不拥有,同一库换参即可打另一棵树。Native 扫描使 Landlock 入口包成为普通发布视图依赖,其可执行文件仍由 Worker 平台实现。CLI 为 `dsh-pack-vfs-image --out <file> [--profile web]`;`apps/web` 的 `build:preview` 在预览壳构建后运行它。
|
||||
|
||||
仓库适配器还声明 `webworker-runtime/tests/fixtures/` 下仅用于 preview 的 fixture(测试前置数据) tree。CLI 会把每套具名 fixture 打成一份独立的确定性 overlay 归档,并写出浏览器可读的 manifest(元数据清单)。Overlay 文件绕过 npm 发布视图和模块可达性排除规则,因此点目录与示例源码会完整保留;其挂载位置仅限 `home/` 与 `workspace/`。`pack.ts` 把它们视为不透明字节;会话与 Workspace 的解释仍归拥有这些格式的运行时包。
|
||||
仓库适配器还声明 `webworker-runtime/tests/fixtures/` 下仅用于 preview 的 fixture(测试前置数据) tree。CLI 会把每套具名 fixture 打成一份独立的确定性 overlay 归档,并写出浏览器可读的 manifest(元数据清单)。Overlay 文件绕过 npm 发布视图和模块可达性排除规则,因此点目录与示例源码会完整保留;其挂载位置仅限 `home/` 与 `workspace/`。`packVfsOverlay` 将所有文件视为不透明字节。
|
||||
|
||||
`packPreviewFixture` 在 Node 中通过 [Session 格式目录](../../session/session-format-catalog/README.zh.md)准备每个 fixture 目录里最新的规范原始 Session 代际。它严格恢复源文件、编码并严格校验当前后继文件,然后将后继文件与未改变的源文件一同打包。如果选中的代际格式错误、不受支持或属于未来版本,构建会拒绝,不回退。临时后继文件在打包后删除。浏览器以写模式打开时即可使用当前数据,无需调用 Node 的迁移校验 Worker。投影缓存仍由其所属包校验;较早代际不能提供当前折叠检查点。
|
||||
|
||||
-----
|
||||
|
||||
@@ -57,6 +59,7 @@ VFS 镜像打包器:把一份合成 profile 变成浏览器 worker 挂载为
|
||||
- **vendored 包源码(`src/*.ts`)被排除**——运行时无人解析它们;未来若有 worker 内源码巡检功能需要专门的 include 规则。
|
||||
- **打包器假定构建产物 `lib/` 是新鲜的**:它从不编译,工作区构建陈旧就会打包陈旧字节。先跑仓库构建。
|
||||
- **CLI 必须位于原有仓库位置**:`dsh-pack-vfs-image` 相对于自身安装文件定位检出目录,并从中读取源码 CLI、配置树和预览 fixture。npm 安装支持参数化的库 API;CLI 和仓库辅助函数需要完整且已构建的 DeepSeek Harness 检出目录。
|
||||
- **Preview Session 输入使用原始 JSONL** — `packPreviewFixture` 拒绝压缩文件与不规范的 Session 代际文件名。通用 overlay 仍逐字节保留输入。
|
||||
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -36,6 +36,8 @@
|
||||
"@deepseek-ai/cordis-plugin-include": "workspace:^",
|
||||
"@deepseek-ai/dsh-experimental-webworker-runtime": "workspace:^",
|
||||
"@deepseek-ai/dsh-home-paths": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-format": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-format-catalog": "workspace:^",
|
||||
"js-yaml": "^4.2.0",
|
||||
"picomatch": "^4.0.4"
|
||||
},
|
||||
|
||||
@@ -14,9 +14,9 @@ import {
|
||||
PREVIEW_FIXTURE_MANIFEST_FILE, PREVIEW_FIXTURE_MANIFEST_VERSION,
|
||||
type PreviewFixtureManifest,
|
||||
} from '@deepseek-ai/dsh-experimental-webworker-runtime'
|
||||
import { packVfsImage, packVfsOverlay } from './pack.ts'
|
||||
import { packVfsImage } from './pack.ts'
|
||||
import {
|
||||
composeProfile, configTrees, describePack, indexWorkspacePackages, previewFixtures,
|
||||
composeProfile, configTrees, describePack, indexWorkspacePackages, packPreviewFixture, previewFixtures,
|
||||
} from './repository.ts'
|
||||
|
||||
/**
|
||||
@@ -66,7 +66,7 @@ const fixtureDirectory = join(dirname(outputFile), 'fixtures')
|
||||
mkdirSync(fixtureDirectory, { recursive: true })
|
||||
const fixtureLines: string[] = []
|
||||
const fixtures = fixtureDefinitions.map((fixture) => {
|
||||
const packed = packVfsOverlay(fixture.trees)
|
||||
const packed = packPreviewFixture(fixture.trees)
|
||||
const file = `fixtures/${fixture.id}.tar.gz`
|
||||
writeFileSync(join(dirname(outputFile), file), packed.image)
|
||||
fixtureLines.push(` fixture overlay ${fixture.id} (${String(packed.image.byteLength)} B compressed)`)
|
||||
|
||||
@@ -11,6 +11,6 @@ export {
|
||||
type ConfigTree, type ImageTree, type PackOptions, type PackOverlayResult, type PackResult,
|
||||
} from './pack.ts'
|
||||
export {
|
||||
composeProfile, configTrees, describePack, indexWorkspacePackages, previewFixtures,
|
||||
composeProfile, configTrees, describePack, indexWorkspacePackages, packPreviewFixture, previewFixtures,
|
||||
type PreviewFixture,
|
||||
} from './repository.ts'
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
/** Node-side Session preparation for browser Preview data overlays. */
|
||||
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { dirname, join, posix } from 'node:path'
|
||||
import { isSessionFormatJsonObject, parseSessionFormatLogFilename, sessionFormatLogFilename } from '@deepseek-ai/dsh-session-format'
|
||||
import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog'
|
||||
import { packVfsOverlay, type ImageTree, type PackOverlayResult } from './pack.ts'
|
||||
import type { ImageFiles } from './transform-image.ts'
|
||||
|
||||
interface PreviewSession {
|
||||
readonly path: string
|
||||
readonly directory: string
|
||||
readonly id: string
|
||||
readonly version: number
|
||||
readonly bytes: Uint8Array
|
||||
}
|
||||
|
||||
/**
|
||||
* Pack Preview data with strictly validated current Session successors prepared in Node.
|
||||
* Committed source bytes remain in the overlay; only the newest canonical raw generation
|
||||
* of each Session is restored. Future, malformed, and unsupported selected data refuse the pack.
|
||||
* @param trees - Ordered source trees under the Preview's home and workspace mounts.
|
||||
* @returns Deterministic overlay containing unchanged inputs and final current successors.
|
||||
*/
|
||||
export function packPreviewFixture(trees: readonly ImageTree[]): PackOverlayResult {
|
||||
const original = packVfsOverlay(trees)
|
||||
const successors = new Map<string, string>()
|
||||
for (const source of selectedSessions(original.files)) {
|
||||
const text = new TextDecoder('utf-8', { fatal: true }).decode(source.bytes)
|
||||
if (!text.endsWith('\n')) throw new Error(`preview fixture: ${source.path} has a torn physical tail`)
|
||||
const [headerLine = '', ...rows] = text.slice(0, -1).split('\n')
|
||||
const header: unknown = JSON.parse(headerLine)
|
||||
if (!isSessionFormatJsonObject(header) || header['version'] !== source.version || header['id'] !== source.id) {
|
||||
throw new Error(`preview fixture: ${source.path} disagrees with its Session header`)
|
||||
}
|
||||
const restore = sessionFormatCatalog.createRestore(header, { recovery: 'strict', validation: 'current' })
|
||||
for (const row of rows) restore.decodeRow(JSON.parse(row))
|
||||
const artifact = restore.finish()
|
||||
if (source.version === sessionFormatCatalog.currentVersion) continue
|
||||
const encodedHeader = sessionFormatCatalog.encodeCurrentHeader(artifact.header, artifact.inheritedEventCount)
|
||||
const encodedEvents = artifact.events.map(event => sessionFormatCatalog.encodeCurrentEvent(event))
|
||||
const verification = sessionFormatCatalog.createRestore(encodedHeader, { recovery: 'strict', validation: 'current' })
|
||||
for (const event of encodedEvents) verification.decodeRow(event)
|
||||
verification.finish()
|
||||
successors.set(
|
||||
posix.join(source.directory, sessionFormatLogFilename(sessionFormatCatalog.currentVersion)),
|
||||
[encodedHeader, ...encodedEvents].map(row => JSON.stringify(row) + '\n').join(''),
|
||||
)
|
||||
}
|
||||
if (successors.size === 0) return original
|
||||
const directory = mkdtempSync(join(tmpdir(), 'dsh-preview-sessions-'))
|
||||
try {
|
||||
for (const [imagePath, content] of successors) {
|
||||
const path = join(directory, imagePath.slice('home/sessions/'.length))
|
||||
mkdirSync(dirname(path), { recursive: true })
|
||||
writeFileSync(path, content, { flag: 'wx' })
|
||||
}
|
||||
return packVfsOverlay([...trees, { mount: 'home/sessions', directory }])
|
||||
} finally {
|
||||
rmSync(directory, { recursive: true, force: true })
|
||||
}
|
||||
}
|
||||
|
||||
function selectedSessions(files: ImageFiles): readonly PreviewSession[] {
|
||||
const selected = new Map<string, PreviewSession>()
|
||||
for (const [path, bytes] of Object.entries(files)) {
|
||||
const match = /^home\/sessions\/[^/]+\/([^/]+)\/(session(?:\..*)?\.jsonl(?:\.zstd)?)$/u.exec(path)
|
||||
if (match === null) continue
|
||||
const version = parseSessionFormatLogFilename(match[2] as string)
|
||||
if (version === undefined) throw new Error(`preview fixture: ${path} must name a canonical raw Session generation`)
|
||||
const directory = posix.dirname(path)
|
||||
const previous = selected.get(directory)
|
||||
if (previous === undefined || previous.version < version) {
|
||||
selected.set(directory, { path, directory, id: match[1] as string, version, bytes })
|
||||
}
|
||||
}
|
||||
return [...selected.values()]
|
||||
}
|
||||
@@ -14,6 +14,8 @@ import { join, relative } from 'node:path'
|
||||
import { DSH_HOME_ENV } from '@deepseek-ai/dsh-home-paths'
|
||||
import type { ConfigTree, ImageTree, PackResult } from './pack.ts'
|
||||
|
||||
export { packPreviewFixture } from './preview.ts'
|
||||
|
||||
/**
|
||||
* Repository directories scanned for workspace and vendored packages. The
|
||||
* image only ever materializes runtime packages, which live here. The Landlock
|
||||
|
||||
@@ -76,6 +76,7 @@ it('loads both tarballs through plain Node and mounts their base image and overl
|
||||
import * as packer from '@deepseek-ai/dsh-experimental-webworker-packer'
|
||||
import * as runtime from '@deepseek-ai/dsh-experimental-webworker-runtime'
|
||||
import * as client from '@deepseek-ai/dsh-experimental-webworker-runtime/client'
|
||||
import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog'
|
||||
for (const name of ['webworker-packer', 'webworker-runtime']) {
|
||||
assert.equal(import.meta.resolve('@deepseek-ai/dsh-experimental-' + name),
|
||||
new URL('./node_modules/@deepseek-ai/dsh-experimental-' + name + '/lib/index.js', import.meta.url).href)
|
||||
@@ -93,6 +94,14 @@ it('loads both tarballs through plain Node and mounts their base image and overl
|
||||
const overlay = packer.packVfsOverlay([{ mount: 'workspace', directory: fileURLToPath(new URL('./overlay', import.meta.url)) }])
|
||||
runtime.loadVfsOverlay(await runtime.inflateImage(overlay.image, 'packed overlay'), '/dsh', vfs)
|
||||
assert.equal(vfs.readFileSync('/dsh/workspace/hello.txt', 'utf8'), 'packed worker pair\\n')
|
||||
mkdirSync('home/sessions/project/preview', { recursive: true })
|
||||
const historical = JSON.stringify({ type: 'session', version: 3, id: 'preview', createdAt: 1, isSeeded: false, delegationDepth: 0 }) + '\\n'
|
||||
writeFileSync('home/sessions/project/preview/session.v3.jsonl', historical)
|
||||
const prepared = packer.packPreviewFixture([{ mount: 'home', directory: fileURLToPath(new URL('./home', import.meta.url)) }])
|
||||
runtime.loadVfsOverlay(await runtime.inflateImage(prepared.image, 'prepared preview'), '/dsh', vfs)
|
||||
assert.equal(vfs.readFileSync('/dsh/home/sessions/project/preview/session.v3.jsonl', 'utf8'), historical)
|
||||
const current = JSON.parse(vfs.readFileSync('/dsh/home/sessions/project/preview/session.v' + sessionFormatCatalog.currentVersion + '.jsonl', 'utf8'))
|
||||
assert.equal(current.version, sessionFormatCatalog.currentVersion)
|
||||
console.log('packed image and overlay mounted')
|
||||
`
|
||||
await writeFile(join(root, 'consumer.mjs'), script)
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog'
|
||||
import { sessionFormatLogFilename } from '@deepseek-ai/dsh-session-format'
|
||||
import { packPreviewFixture } from '../src/preview.ts'
|
||||
import { packVfsOverlay } from '../src/pack.ts'
|
||||
import { previewFixtures } from '../src/repository.ts'
|
||||
|
||||
const repoRoot = fileURLToPath(new URL('../../../../', import.meta.url))
|
||||
const roots: string[] = []
|
||||
const currentVersion = sessionFormatCatalog.currentVersion
|
||||
const text = (bytes: Uint8Array | undefined): string => new TextDecoder().decode(bytes)
|
||||
|
||||
afterEach(() => {
|
||||
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
function source() {
|
||||
const root = mkdtempSync(join(tmpdir(), 'dsh-preview-source-'))
|
||||
roots.push(root)
|
||||
const session = join(root, 'sessions', 'project', 'example')
|
||||
mkdirSync(session, { recursive: true })
|
||||
return {
|
||||
trees: [{ mount: 'home', directory: root }],
|
||||
put(version: number, body = '', overrides: Record<string, unknown> = {}) {
|
||||
const path = join(session, sessionFormatLogFilename(version))
|
||||
const header = { type: 'session', version, id: 'example', createdAt: 1, isSeeded: false, delegationDepth: 0, ...overrides }
|
||||
writeFileSync(path, JSON.stringify(header) + '\n' + body)
|
||||
return path
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function restore(bytes: Uint8Array | undefined) {
|
||||
const [header, ...rows] = text(bytes).trimEnd().split('\n').map(row => JSON.parse(row) as unknown)
|
||||
const reader = sessionFormatCatalog.createRestore(header, { recovery: 'strict', validation: 'current' })
|
||||
for (const row of rows) reader.decodeRow(row)
|
||||
return reader.finish()
|
||||
}
|
||||
|
||||
describe('Node preparation of Preview Session data', () => {
|
||||
it('packs strictly migrated current successors beside every unchanged example file', () => {
|
||||
const fixture = previewFixtures(repoRoot)[0]!
|
||||
const original = packVfsOverlay(fixture.trees)
|
||||
const projected = packPreviewFixture(fixture.trees)
|
||||
const currentFiles = Object.keys(projected.files).filter(path => path.endsWith('/' + sessionFormatLogFilename(currentVersion)))
|
||||
expect(currentFiles).toHaveLength(3)
|
||||
for (const [path, bytes] of Object.entries(original.files)) expect(projected.files[path], path).toEqual(bytes)
|
||||
for (const path of currentFiles) {
|
||||
const sourcePath = path.replace(/\/session\.v\d+\.jsonl$/u, '/session.v3.jsonl')
|
||||
expect(restore(projected.files[path])).toEqual(restore(original.files[sourcePath]))
|
||||
expect(JSON.parse(text(projected.files[path]).split('\n')[0]!)).toMatchObject({ version: currentVersion })
|
||||
}
|
||||
expect(packPreviewFixture(fixture.trees).image).toEqual(projected.image)
|
||||
expect(packVfsOverlay(fixture.trees).files).toEqual(original.files)
|
||||
})
|
||||
|
||||
it('uses the highest generation and leaves an already-current image byte-identical', () => {
|
||||
const fixture = source()
|
||||
const older = fixture.put(3, '{invalid historical data}\n')
|
||||
fixture.put(currentVersion)
|
||||
const original = packVfsOverlay(fixture.trees)
|
||||
expect(packPreviewFixture(fixture.trees).image).toEqual(original.image)
|
||||
expect(text(original.files['home/sessions/project/example/session.v3.jsonl'])).toBe(readFileSync(older, 'utf8'))
|
||||
})
|
||||
|
||||
it('keeps an overlay without Sessions opaque', () => {
|
||||
const fixture = source()
|
||||
expect(packPreviewFixture(fixture.trees)).toEqual(packVfsOverlay(fixture.trees))
|
||||
})
|
||||
|
||||
it.each(['future', 'corrupt', 'unknown-required'] as const)('refuses selected %s data without falling back or changing inputs', (mode) => {
|
||||
const fixture = source()
|
||||
fixture.put(3)
|
||||
const path = fixture.put(mode === 'future' ? currentVersion + 1 : currentVersion,
|
||||
mode === 'corrupt' ? '{invalid JSON}\n'
|
||||
: mode === 'unknown-required' ? JSON.stringify({ type: 'external/required', seq: 0, time: 1, data: null }) + '\n' : '')
|
||||
const before = readFileSync(path)
|
||||
expect(() => packPreviewFixture(fixture.trees)).toThrow()
|
||||
expect(readFileSync(path)).toEqual(before)
|
||||
})
|
||||
|
||||
it.each([{ version: 2 }, { id: 'other' }])('refuses selected metadata that disagrees with its location: %j', (overrides) => {
|
||||
const fixture = source()
|
||||
fixture.put(3, '', overrides)
|
||||
expect(() => packPreviewFixture(fixture.trees)).toThrow('disagrees with its Session header')
|
||||
})
|
||||
|
||||
it('refuses a torn physical tail and compressed Session generations', () => {
|
||||
const fixture = source()
|
||||
const path = fixture.put(3)
|
||||
const bytes = readFileSync(path)
|
||||
writeFileSync(path, bytes.subarray(0, -1))
|
||||
expect(() => packPreviewFixture(fixture.trees)).toThrow('torn physical tail')
|
||||
writeFileSync(path, bytes)
|
||||
writeFileSync(join(dirname(path), sessionFormatLogFilename(currentVersion) + '.zstd'), 'compressed data')
|
||||
expect(() => packPreviewFixture(fixture.trees)).toThrow('canonical raw Session generation')
|
||||
})
|
||||
})
|
||||
@@ -19,6 +19,12 @@
|
||||
},
|
||||
{
|
||||
"path": "../../util/home-paths"
|
||||
},
|
||||
{
|
||||
"path": "../../session/session-format"
|
||||
},
|
||||
{
|
||||
"path": "../../session/session-format-catalog"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/experimental/webworker-runtime/README.md
|
||||
README.md: 724e47a004fac83fe0f19a588faf0c227d843232
|
||||
README.zh.md: 4e88b3cf10f98a423268c60b69269d2550d2c1f9
|
||||
README.md: de423c3606f1fd24582b6db1d3ffd011bfd4e632
|
||||
README.zh.md: ca5ca793e7c56b1b1fccb65b356f3c98c03cac2b
|
||||
|
||||
@@ -32,7 +32,7 @@ Three build artifacts and one source-owned process layer:
|
||||
- **`src/shell/` (the worker's own process layer)** — a browser worker cannot fork, so `node:child_process` is not a stub but an implementation: `spawn` starts the command in its own Web Worker — this same bundle, told by its first frame to be a shell process — and reports it through the `ChildProcess` surface the subprocess service consumes. The command runs off the host's thread, `SIGKILL` terminates it whatever it is doing, and it reaches the VFS only by message (the host serves those frames). Worker platform executables preserve native-package protocols such as Landlock without replacing their JavaScript packages or coupling their implementations to `node:child_process`; ordinary commands use the package's evaluator and coreutils command table. The grammar is `@yarnpkg/parsers`' `parseShell`, while `execSync`/`fork` still refuse because they need a real process.
|
||||
- **`lib/client.js` (page half)** — startup has two independent stages. `chooseWorkerHostSource({ image?, fixtureManifest? })` optionally owns the boot barrier and fixture manifest: without `preview-fixture` it waits at the source chooser, while a valid query selects directly; either path returns ordered overlays. `connectWorkerHost(worker, { image?, overlays? })` remains the public base-runtime connector; callers that skip the chooser get an empty overlay list. `apps/web` invokes both and supplies its statically bundled Worker. The opening `init` frame carries the base and ordered overlay URLs, the boot payload delivers the structured index-injection table, and `applyIndexInjections` executes it before the shell entry runs. Script preload rows are advisory and skipped because `/plugins` resources resolve only through the tunnel; `loadBundle` fetches each combo on first demand, embeds its tunnel-only source map as a Base64 data URL, and executes the script as a Blob. The tunnel also exposes fetch-shaped transport, the independent file-upload carrier, and the API client. Request frames preserve Blob bodies through structured clone and transfer `ReadableStream<Uint8Array>` ownership. The Host Worker streams both forms into the route, so neither browser thread creates a complete byte array for a generic-file upload.
|
||||
|
||||
Acceptance lives in `apps/web/tests/preview-boot.e2e.ts`, which serves the real built pages and drives the pre-boot chooser plus Worker activation in headless Chromium. The empty selection exercises first-run startup. The `vfs-example` overlay supplies ordinary workspace files and plaintext persistence artifacts for cold Workspace/Session discovery, tool presentation, subagent navigation, and history paging without a model request. The fixture generator owns current-generation logs and the projection cache; committed predecessor logs remain byte-identical alongside them. The chooser reserves WebFS as a separate user-authorized source; that provider does not read the built-in fixture.
|
||||
Acceptance lives in `apps/web/tests/preview-boot.e2e.ts`, which serves the real built pages and drives the pre-boot chooser plus Worker activation in headless Chromium. The empty selection exercises first-run startup. The `vfs-example` overlay supplies ordinary workspace files and plaintext persistence artifacts for cold Workspace/Session discovery, tool presentation, subagent navigation, and history paging without a model request. The [packer](../webworker-packer/README.md) prepares current-generation successors from committed historical logs in Node; those source logs remain byte-identical in the overlay. The fixture generator owns the acceptance test's current projection cache. The chooser reserves WebFS as a separate user-authorized source; that provider does not read the built-in fixture.
|
||||
|
||||
The [built-bundle import sweep](tests/compile/transform-corpus-check.ts) checks bare Node imports after the library build. Its Dockkit exception accepts Node’s unknown-`.css`-extension refusal for any stylesheet rather than one pinned path; another extension, error code, or message, and an unexpectedly successful exempt import, remain errors. See the [stylesheet exemption decision](../../../.agents/notes/implemented/bug-fix/2026-09-10-built-bundle-css-exemption.md).
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ kind: "package-library"
|
||||
- **`src/shell/`(worker 自己的进程层)**——浏览器 worker 无法 fork,所以 `node:child_process` 不是 stub 而是实现:`spawn` 把命令放进它自己的 Web Worker——就是这同一个束,由首帧告诉它「你是 shell 进程」——并以 subprocess 服务消费的 `ChildProcess` 面报告结果。命令不占宿主线程,`SIGKILL` 不管它在干什么都能终止它,而它只能靠消息触达 VFS(由宿主应答这些帧)。Worker 平台 executable 在不替换 JavaScript 包、也不把具体实现耦合进 `node:child_process` 的情况下保持 Landlock 等 native 包协议;普通命令使用本包的求值器与 coreutils 命令表。语法来自 `@yarnpkg/parsers` 的 `parseShell`,而 `execSync`/`fork` 依然拒绝,因为它们需要真进程。
|
||||
- **`lib/client.js`(页面半)**——启动分为相互独立的两段。`chooseWorkerHostSource({ image?, fixtureManifest? })` 可选地拥有 boot barrier 与 fixture manifest:没有 `preview-fixture` 时停在来源选择面板,合法 query 则直接选择;两条路径都返回按序排列的 overlays。`connectWorkerHost(worker, { image?, overlays? })` 仍是公开的基础运行态连接器;调用方跳过选择器时 overlay 列表为空。`apps/web` 调用这两段并提供静态打包的 Worker。开局 `init` 帧携带基础镜像与按序排列的 overlay URL,boot 载荷送达结构化 index 注入表,`applyIndexInjections` 在壳入口运行前逐行执行。脚本 preload 行只是提示,因此会被跳过:`/plugins` 资源只能经 tunnel 解析,`loadBundle` 会在首次需要时获取 combo、把仅 tunnel 可达的 sourcemap 内嵌为 Base64 data URL,再以 Blob 执行脚本。Tunnel 还暴露 fetch 形式的传输、独立文件上传载体和 API 客户端。请求帧通过结构化克隆保留 Blob 请求体,并转移 `ReadableStream<Uint8Array>` 的所有权。Host Worker 将两种请求体都逐块送入路由,因此通用文件上传不会在任何浏览器线程创建完整字节数组。
|
||||
|
||||
验收在 `apps/web/tests/preview-boot.e2e.ts`:静态服务真实构建页面,在 headless Chromium 里驱动 pre-boot 选择面板与 Worker 激活。空白选择验证首次启动;`vfs-example` overlay 提供普通 workspace 文件与明文 persistence 产物,无需模型请求即可验证 Workspace/Session 冷发现、工具呈现、subagent 导航和历史分页。fixture 生成器负责当前代日志与投影缓存;已提交的前代日志逐字节保持不变,与它们并存。选择面板为 WebFS 保留独立的用户授权来源;该 provider 不读取内置 fixture。
|
||||
验收在 `apps/web/tests/preview-boot.e2e.ts`:静态服务真实构建页面,在 headless Chromium 里驱动 pre-boot 选择面板与 Worker 激活。空白选择验证首次启动;`vfs-example` overlay 提供普通 workspace 文件与明文 persistence 产物,无需模型请求即可验证 Workspace/Session 冷发现、工具呈现、subagent 导航和历史分页。[打包器](../webworker-packer/README.zh.md)在 Node 中根据已提交的历史日志准备当前代后继文件;这些源日志在 overlay 中逐字节保持不变。fixture 生成器负责验收测试的当前投影缓存。选择面板为 WebFS 保留独立的用户授权来源;该 provider 不读取内置 fixture。
|
||||
|
||||
[已构建 bundle 导入检查](tests/compile/transform-corpus-check.ts)在库构建后检查裸 Node 导入。Dockkit 例外接受 Node 针对任意样式表报告的未知 `.css` 扩展名错误,而不限定某一固定路径;其他扩展名、错误码或消息,以及意外成功的豁免导入,仍然报错。参见 [样式表豁免决策](../../../.agents/notes/implemented/bug-fix/2026-09-10-built-bundle-css-exemption.zh.md)。
|
||||
|
||||
|
||||
Generated
+6
@@ -6495,6 +6495,12 @@ importers:
|
||||
'@deepseek-ai/dsh-home-paths':
|
||||
specifier: workspace:^
|
||||
version: link:../../util/home-paths
|
||||
'@deepseek-ai/dsh-session-format':
|
||||
specifier: workspace:^
|
||||
version: link:../../session/session-format
|
||||
'@deepseek-ai/dsh-session-format-catalog':
|
||||
specifier: workspace:^
|
||||
version: link:../../session/session-format-catalog
|
||||
js-yaml:
|
||||
specifier: ^4.2.0
|
||||
version: 4.3.1
|
||||
|
||||
Reference in New Issue
Block a user