🧪 Pin fork benchmark heap for consistent memory measurements

2026-11-09 20:07:43 +08:00 · 2026-10-22 14:37:12 +08:00
parent 2faf547465
commit e689139284
7 changed files with 22 additions and 9 deletions
@@ -18,11 +18,11 @@
  en: 1310a89c351e5bfe
  zh: e6386ae10b893c2d
 /agent-note-performance-baselines-for-large-session-corpora/decision/measured-endpoints:
-  en: 11937dfa4a349a19
-  zh: b91a136890fe92e8
+  en: add81088ce51033f
+  zh: c7065dc3052648fa
 /agent-note-performance-baselines-for-large-session-corpora/calibration-evidence:
-  en: f934c95748dcbfe0
-  zh: df6a68286c21510f
+  en: adb815c213373d8e
+  zh: 44d4c6752a4df51f
 /agent-note-performance-baselines-for-large-session-corpora/alternatives-considered:
  en: 4eaa4666262a1aee
  zh: 7e2eb44b94bad05d
@@ -47,6 +47,8 @@ The shared corpus and projection-list controller fixture awaits controller activ

 The file must finish within five minutes on standard hosted CI, including seeding, and its last case asserts that limit. Search over 1,000 Sessions and one fork of the longest Session take about 170 s of it, so each has one sample. The list corpus is the largest that fits the remainder. Budgets enforce medians of list boot, first, and repeat, and of the fork median over strata and rank 990; search and the longest fork enforce their single samples. Search and fork use separate 1,000-Session roots, so fork children never reach the search index. Rank 988 is the highest top-level rank at the measured p99 (7,229 events); forking a subagent child adds a lineage lookup, and rank 990 already falls in the 99.3% anchor. Peak RSS of the list, search, and fork processes has a budget with the shared 1.25 headroom and no time scale. Each child's deadline equals the file limit, and each test's deadline outlasts its sequential children, so a failed case reaps every child before the scratch root is removed.

+Fork workers use a 512 MiB V8 old-space limit so every runner measures the same heap constraint. The corpus, timed endpoints, and whole-process peak RSS budget stay unchanged; native buffers still count toward RSS. The longest fork retains about 193 MiB after collection, so the limit leaves room for live objects while forcing collection before a larger host-selected heap delays it. This measures a bounded-heap workload, not the default-heap peak of every deployment.
+
 ## Calibration evidence

 Budgets use standard two-CPU hosted CI expectations (Linux x64, Node 24.21, PR #5403), rounded above the higher recorded median across the AMD EPYC 7763 and EPYC 9V74 runners and multiplied by the shared 1.25 headroom. Samples varied by up to 15% within one run; the runner models differed by up to 25% per endpoint. The hosted-to-reference ratio is 2.4 to 3.4, above the shared reference scale of 2, so reference-machine scaling would reject ordinary hosted runs.
@@ -67,6 +69,8 @@ With seeding in one process, the complete file took 294.5 s on the EPYC 7763 run

 On Apple M5 Pro with Node 26.5, isolated runs measured list 1,000 at 311–358 ms first, list 5,000 at 1,364–1,603 ms first, search at 57 s first, and fork of the longest Session at 6.0–6.4 s. A 10,000-Session list took 3.2–5.4 s for the first call with 1.9 GB peak RSS.

+Node 24.22 hosted runs of the unchanged longest-fork workload recorded 696 MiB peak RSS on EPYC 9V74 and 1,077.7 MiB on EPYC 7763, against the same 938 MiB budget. On Apple M5 Pro with Node 24.19, two default-heap runs retained 192.8–193.4 MiB and peaked at 770.7–812.6 MiB in 2.416–2.447 s; two 512 MiB old-space runs retained the same range and peaked at 764.2–764.7 MiB in 2.487–2.528 s. The fixed limit controls the heap policy without raising the RSS or timing budgets.
+
 ## Alternatives considered

 - **Sanitized copies of real Sessions.** Rejected: benchmarks never use recorded user material.
@@ -47,6 +47,8 @@ worker 是编译后的纯 Node 进程，每次都针对已播种根目录全新

 此文件（包括播种）必须在标准托管 CI 上五分钟内完成，其最后一个用例断言该上限。1,000 个 Session 的搜索和最长 Session 的一次 fork 约占其中 170 s，因此两者各取一个样本。列表语料是剩余时间内能容纳的最大规模。预算约束列表启动、首次和重复的中位数，以及 fork 分层中位数和排名 990 的中位数；搜索和最长 fork 约束各自的单个样本。搜索和 fork 使用各自独立的 1,000 个 Session 根目录，因此 fork 子会话不会进入搜索索引。排名 988 是实测 p99（7,229 个事件）上最高的顶层排名；fork subagent 子会话会增加一次谱系查找，而排名 990 已落入 99.3% 锚点。列表、搜索和 fork 进程的峰值 RSS 设有预算，使用共享的 1.25 余量，不做时间缩放。每个子进程的截止时间等于文件上限，每个测试的截止时间长于其顺序子进程之和，因此失败的用例会在删除临时根目录之前回收所有子进程。

+Fork worker 使用 512 MiB 的 V8 old-space 上限，让各运行器在相同的堆约束下测量。语料、计时终点和整个进程的峰值 RSS 预算保持不变，原生缓冲区仍计入 RSS。最长 fork 回收后保留约 193 MiB，因此该上限为存活对象保留空间，同时避免较大的宿主默认堆延迟回收。该用例测量受限堆下的工作负载，不代表每种部署默认堆下的峰值。
+
 ## 校准证据

 预算使用标准双 CPU 托管 CI 的期望值（Linux x64、Node 24.21、PR #5403），取高于 AMD EPYC 7763 与 EPYC 9V74 运行器中较高记录中位数的整数，再乘以共享的 1.25 余量。同一次运行内的样本差异最多 15%；两种运行器型号在单个端点上最多相差 25%。托管与参考机器的比值为 2.4 到 3.4，高于共享的参考缩放系数 2，因此按参考机器缩放会拒绝正常的托管运行。
@@ -67,6 +69,8 @@ worker 是编译后的纯 Node 进程，每次都针对已播种根目录全新

 在 Apple M5 Pro 和 Node 26.5 上的隔离运行中，1,000 个 Session 的首次列表为 311–358 ms，5,000 个 Session 的首次列表为 1,364–1,603 ms，首次搜索为 57 s，fork 最长 Session 为 6.0–6.4 s。10,000 个 Session 的首次列表耗时 3.2–5.4 s，峰值 RSS 为 1.9 GB。

+未改动的最长 fork 工作负载在 Node 24.22 托管运行中，EPYC 9V74 的峰值 RSS 为 696 MiB，EPYC 7763 为 1,077.7 MiB，使用的预算均为 938 MiB。在 Apple M5 Pro、Node 24.19 上，两次默认堆运行回收后保留 192.8–193.4 MiB，峰值为 770.7–812.6 MiB，耗时 2.416–2.447 s；两次 512 MiB old-space 运行保留量相同，峰值为 764.2–764.7 MiB，耗时 2.487–2.528 s。固定上限用于控制堆策略，没有提高 RSS 或耗时预算。
+
 ## 考虑过的替代方案

 - **脱敏复制真实 Session。** 已否决：基准从不使用录制的用户材料。
@@ -12,8 +12,8 @@
  en: 66a769ce60d49ff3
  zh: 7e0bfa7c598157fa
 /session-corpus-benchmarks/run:
-  en: 2560e832041ae481
-  zh: 2fbbff71135e52ab
+  en: c770082e58aa8194
+  zh: 339d659595a1f97d
 /session-corpus-benchmarks/measurements:
  en: c357c231c3bfb588
  zh: 3d901a2caccf2587
@@ -18,7 +18,7 @@ Measure the Web Host's Session list, content search, and fork over synthetic cor

 From the repository root, build the libraries and workers with `pnpm run build:bench`, then run `pnpm exec vitest run --config vitest.bench.config.ts benchmarks/session-corpus/session-corpus.bench.ts`. Do not overlap timing runs with builds or other benchmarks. Seeding writes about 1.5 GB of Zstandard logs under a private temporary root, which the test removes after success or failure.

-The test reports every fresh-process sample, the CPU model, available parallelism, platform, and Node version, and enforces median budgets. Its last case fails when the file, including seeding, exceeds five minutes. A failed worker reports its exit, signal, timeout, and stderr tail. The required benchmark lane discovers this file automatically.
+The test reports every fresh-process sample, the CPU model, available parallelism, platform, and Node version, and enforces median budgets. Its last case fails when the file, including seeding, exceeds five minutes. Fork workers use a fixed V8 heap limit; peak RSS still measures the whole process. A failed worker reports its exit, signal, timeout, and stderr tail. The required benchmark lane discovers this file automatically.

 <a id="measurements"></a>

@@ -18,7 +18,7 @@

 在仓库根目录使用 `pnpm run build:bench` 构建库和 worker，然后运行 `pnpm exec vitest run --config vitest.bench.config.ts benchmarks/session-corpus/session-corpus.bench.ts`。不要让计时运行与构建或其他基准重叠。播种过程会在私有临时根目录下写入约 1.5 GB 的 Zstandard 日志，测试成功或失败后都会删除该目录。

-测试报告每个新进程样本、CPU 型号、可用并行度、平台和 Node 版本，并约束中位数预算。当整个文件（包括播种）超过五分钟时，最后一个用例失败。worker 失败时报告退出状态、信号、超时和 stderr 末尾。必需基准通道自动发现此文件。
+测试报告每个新进程样本、CPU 型号、可用并行度、平台和 Node 版本，并约束中位数预算。当整个文件（包括播种）超过五分钟时，最后一个用例失败。Fork worker 使用固定的 V8 堆上限；峰值 RSS 仍测量整个进程。worker 失败时报告退出状态、信号、超时和 stderr 末尾。必需基准通道自动发现此文件。

 <a id="measurements"></a>

@@ -29,6 +29,8 @@ const CORPUS = { search: 1_000, fork: 1_000, list: 3_000 } as const
 const FILE_LIMIT_MS = 300_000
 /** Per-child deadline: an endpoint slower than its budget still reports a measurement instead of a kill. */
 const WORKER_TIMEOUT_MS = FILE_LIMIT_MS
+/** Fixed old-space capacity makes fork RSS comparable across hosted V8 heap defaults. */
+const FORK_HEAP_LIMIT_MB = 512
 /** Fresh processes per list and fork-strata scenario; the median enforces each budget. */
 const ATTEMPTS = { list: 3, fork: 3 } as const
 /** Concurrent anchor-preparation processes; standard hosted runners have two CPUs. */
@@ -119,7 +121,10 @@ function serialDeadline(workers: number): number {
 }

 async function run<Report extends SessionCorpusReport>(args: readonly string[]): Promise<Report> {
-  const outcome = await runBuiltBenchmarkWorker<Report>({ worker: WORKER, args, timeoutMs: WORKER_TIMEOUT_MS, exposeGc: true })
+  const outcome = await runBuiltBenchmarkWorker<Report>({
+    worker: WORKER, args, timeoutMs: WORKER_TIMEOUT_MS, exposeGc: true,
+    ...(args[1] === 'fork' ? { heapLimitMb: FORK_HEAP_LIMIT_MB } : {}),
+  })
  if (outcome.report === undefined) {
    const stderr = outcome.stderr.trim().split('\n').slice(-20).join('\n')
    throw new Error(`session-corpus worker ${args[1] ?? ''} failed: exit=${String(outcome.exitCode)}, `