mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-11-09 20:07:43 +08:00
test: calibrate npm layout budget for the expanded release graph
This commit is contained in:
Yichen Jiang
+6
-6
@@ -9,14 +9,14 @@
|
||||
en: 46916488210ba493
|
||||
zh: 98107c3805a22b2c
|
||||
/agent-note-dual-release-install-layout-work-budget/decision:
|
||||
en: c432b68c4ad1b3c1
|
||||
zh: a11e958fa68dc2f3
|
||||
en: b900469750b27f81
|
||||
zh: d5380d7751332e9c
|
||||
/agent-note-dual-release-install-layout-work-budget/what-the-resolution-costs:
|
||||
en: 36d3e3ef39cbba2f
|
||||
zh: 2480acf6260817b3
|
||||
en: 463df1fae15ae2fc
|
||||
zh: 733c9ddb4900e3fc
|
||||
/agent-note-dual-release-install-layout-work-budget/alternatives-considered:
|
||||
en: 29987b7634c84d28
|
||||
zh: a1469e4b6aaa46e2
|
||||
/agent-note-dual-release-install-layout-work-budget/consequences:
|
||||
en: 8dffc0c3fa699980
|
||||
zh: 99ec25001e5aa716
|
||||
en: 4226e2216abe042d
|
||||
zh: db8f308ef40f47e9
|
||||
|
||||
+5
-3
@@ -10,14 +10,16 @@ The `Dependency layout` lane packs two incompatible synthetic DSH releases into
|
||||
|
||||
## Decision
|
||||
|
||||
`scripts/verify-npm-install-layout.ts` decides growth with `assertResolutionWorkBudget`, which bounds `dshPackagesPerVersion * checkedDshEdges` against `MAX_RESOLUTION_WORK_UNITS = 875_000`. Both counts come from the summary `assertDualDshInstallLayout` returns, so the budget applies to the graph npm actually produced rather than to a prediction of it. A graph above the budget fails with its own counts, the budget, and the instruction to measure the new resolution cost and raise the constant.
|
||||
`scripts/verify-npm-install-layout.ts` decides growth with `assertResolutionWorkBudget`, which bounds `dshPackagesPerVersion * checkedDshEdges` against `MAX_RESOLUTION_WORK_UNITS = 1_134_400`. Both counts come from the summary `assertDualDshInstallLayout` returns, so the budget applies to the graph npm actually produced rather than to a prediction of it. A graph above the budget fails with its own counts, the budget, and the instruction to measure the new resolution cost and raise the constant.
|
||||
|
||||
The npm child keeps a wall-clock limit, `NPM_HANG_GUARD_MS`, derived as `4 * MAX_RESOLUTION_WORK_UNITS * SECONDS_PER_WORK_UNIT` (910 s). It is a hang guard, not the growth criterion: a runner up to four times slower than the measured host cannot decide the gate's outcome, and deriving it from the budget keeps the two values from drifting apart.
|
||||
The npm child keeps a wall-clock limit, `NPM_HANG_GUARD_MS`, derived as `4 * MAX_RESOLUTION_WORK_UNITS * SECONDS_PER_WORK_UNIT` (about 1,180 s). It is a hang guard, not the growth criterion: a runner up to four times slower than the measured host cannot decide the gate's outcome, and deriving it from the budget keeps the two values from drifting apart.
|
||||
|
||||
This restores the [published dependency faces](2026-08-26-published-dependency-faces.md) decision that `verify-npm-install-layout` does not enforce resolver duration; the 300 s deadline had contradicted it.
|
||||
|
||||
## What the resolution costs
|
||||
|
||||
Measured 2026-10-26 on an idle Apple M5 Pro with Node 24.17.0 and npm 11.13.0, using `buildDualDshRegistry`, `resolveNpmPackageLock`, and `assertDualDshInstallLayout` sequentially with fresh temporary consumers: the #5960 graph (315 packages per release, 2782 internal edges, 876,330 work units) resolved in 75.27 s; the #5961 graph (320 packages, 2836 edges, 907,520 units) resolved in 71.03 s, with a second verification at 97.73 s. All placements passed, shared one Cordis installation, and requested no archives. The measurements calibrate graph admission and do not establish a runtime improvement. The current budget is 1.25 times its 907,520 units; the earlier, slower per-unit timing remains the conservative hang-guard calibration.
|
||||
|
||||
Measured 2026-09-28 on an idle M-series host, against `origin/master` (277 DSH packages per release, 2524 internal edges), by running the lane's own npm invocation with `--timing` and `--cpu-prof`:
|
||||
|
||||
| Phase | Cost |
|
||||
@@ -47,4 +49,4 @@ Repeated runs of the same graph on one host span 137.8 s to 204.8 s of npm time,
|
||||
|
||||
## Consequences
|
||||
|
||||
The lane's failure now depends on the graph, so a pull request that adds a few DSH packages passes on any runner, and a graph that grows the packages-times-edges product by more than about a quarter — about 12% in packages and internal edges together — fails deterministically with its measured counts. The budget is a ratchet: raising it is a deliberate edit that must come with a new resolution measurement, and the constant's comment records the counts it was derived from. A graph inside the budget still spends its full resolution time, because the change moves the decision and not the work; nothing here reduces the 153 s npm needs to place the dual release. A genuine npm hang now consumes up to 910 s of the lane before failing, where the old deadline capped it at 300 s. Growth that changes the shape of the graph without changing packages times edges stays inside the budget and is caught only if the hang guard expires.
|
||||
The lane's failure now depends on the graph, so a pull request that adds a few DSH packages passes on any runner, and a graph that grows the packages-times-edges product by more than about a quarter — about 12% in packages and internal edges together — fails deterministically with its measured counts. The budget is a ratchet: raising it is a deliberate edit that must come with a new resolution measurement, and the constant's comment records the counts it was derived from. A graph inside the budget still spends its full resolution time, because the change moves the decision and not the work; nothing here reduces the 153 s npm needs to place the dual release. A genuine npm hang now consumes up to about 1,180 s of the lane before failing, where the old deadline capped it at 300 s. Growth that changes the shape of the graph without changing packages times edges stays inside the budget and is caught only if the hang guard expires.
|
||||
|
||||
+5
-3
@@ -10,14 +10,16 @@ Status: implemented
|
||||
|
||||
## Decision
|
||||
|
||||
`scripts/verify-npm-install-layout.ts` 用 `assertResolutionWorkBudget` 判定规模增长，它以 `dshPackagesPerVersion * checkedDshEdges` 为工作量单位，与 `MAX_RESOLUTION_WORK_UNITS = 875_000` 比较。两个计数都来自 `assertDualDshInstallLayout` 返回的汇总，因此预算作用于 npm 实际产出的图，而不是对图的预测。超过预算的图会带着自己的计数、预算值，以及「测量新的解析开销并调高该常量」的指示失败。
|
||||
`scripts/verify-npm-install-layout.ts` 用 `assertResolutionWorkBudget` 判定规模增长，它以 `dshPackagesPerVersion * checkedDshEdges` 为工作量单位，与 `MAX_RESOLUTION_WORK_UNITS = 1_134_400` 比较。两个计数都来自 `assertDualDshInstallLayout` 返回的汇总，因此预算作用于 npm 实际产出的图，而不是对图的预测。超过预算的图会带着自己的计数、预算值，以及「测量新的解析开销并调高该常量」的指示失败。
|
||||
|
||||
npm 子进程仍保留墙钟上限 `NPM_HANG_GUARD_MS`，由 `4 * MAX_RESOLUTION_WORK_UNITS * SECONDS_PER_WORK_UNIT` 推导（910 秒）。它是挂起保护，不是增长判据：比实测主机慢四倍以内的 runner 无法决定该门禁的结果，而由预算推导可避免两个数值彼此漂移。
|
||||
npm 子进程仍保留墙钟上限 `NPM_HANG_GUARD_MS`，由 `4 * MAX_RESOLUTION_WORK_UNITS * SECONDS_PER_WORK_UNIT` 推导（约 1,180 秒）。它是挂起保护，不是增长判据：比实测主机慢四倍以内的 runner 无法决定该门禁的结果，而由预算推导可避免两个数值彼此漂移。
|
||||
|
||||
这恢复了[发布依赖门面](2026-08-26-published-dependency-faces.zh.md)中「`verify-npm-install-layout` 不限制 resolver 耗时」的决定；300 秒期限与该决定相矛盾。
|
||||
|
||||
## What the resolution costs
|
||||
|
||||
2026-10-26 在空闲 Apple M5 Pro 上使用 Node 24.17.0 与 npm 11.13.0，通过 `buildDualDshRegistry`、`resolveNpmPackageLock` 和 `assertDualDshInstallLayout`，对独立的新建临时消费者目录依次测量：#5960 的依赖图（每个发布 315 个包、2782 条内部边、876,330 个工作量单位）用 75.27 秒解析；#5961 的依赖图（320 个包、2836 条边、907,520 个工作量单位）用 71.03 秒解析，第二次验证耗时 97.73 秒。所有布局校验均通过，共享一个 Cordis 安装，未请求包归档。这些测量用于校准依赖图的准入规模，不代表运行时性能提升。当前预算为其 907,520 个工作量单位的 1.25 倍；挂起保护仍采用此前较慢的单位工作量耗时作为保守校准。
|
||||
|
||||
2026-09-28 在空闲的 M 系列主机上，针对 `origin/master`（每个发布 277 个 DSH 包、2524 条内部边），用 `--timing` 和 `--cpu-prof` 运行 lane 自身的 npm 调用测得：
|
||||
|
||||
| 阶段 | 开销 |
|
||||
@@ -47,4 +49,4 @@ npm 子进程仍保留墙钟上限 `NPM_HANG_GUARD_MS`，由 `4 * MAX_RESOLUTION
|
||||
|
||||
## Consequences
|
||||
|
||||
lane 的失败判定如今取决于依赖图，因此新增少量 DSH 包的 pull request 在任何 runner 上都能通过，而「包数乘边数」增长超过约四分之一的图——即包数与内部边数同步增长约 12%——会带着实测计数确定性地失败。预算是棘轮：调高它是一次刻意修改，必须附带新的解析实测，该常量注释里记录了推导所用的计数。预算内的图仍会花掉完整的解析时间，因为这次改动移动的是判定位置而不是工作本身；这里没有任何东西能减少 npm 摆放双发布所需的 153 s。真正挂起的 npm 现在会消耗该 lane 最多 910 秒才失败，而旧期限把它限制在 300 秒。若图的增长改变了形状却没有改变「包数乘边数」，它仍留在预算内，只有在挂起保护到期时才被发现。
|
||||
lane 的失败判定如今取决于依赖图，因此新增少量 DSH 包的 pull request 在任何 runner 上都能通过，而「包数乘边数」增长超过约四分之一的图——即包数与内部边数同步增长约 12%——会带着实测计数确定性地失败。预算是棘轮：调高它是一次刻意修改，必须附带新的解析实测，该常量注释里记录了推导所用的计数。预算内的图仍会花掉完整的解析时间，因为这次改动移动的是判定位置而不是工作本身；这里没有任何东西能减少 npm 摆放双发布所需的 153 s。真正挂起的 npm 现在会消耗该 lane 最多约 1,180 秒才失败，而旧期限把它限制在 300 秒。若图的增长改变了形状却没有改变「包数乘边数」，它仍留在预算内，只有在挂起保护到期时才被发现。
|
||||
|
||||
@@ -118,12 +118,12 @@ describe('npm install layout verifier', () => {
|
||||
|
||||
describe('resolution work budget', () => {
|
||||
it('accepts the measured dual-release graph', () => {
|
||||
expect(assertResolutionWorkBudget({ dshPackagesPerVersion: 277, checkedDshEdges: 2524 }))
|
||||
.toBe(699_148)
|
||||
expect(assertResolutionWorkBudget({ dshPackagesPerVersion: 320, checkedDshEdges: 2836 }))
|
||||
.toBe(907_520)
|
||||
})
|
||||
|
||||
it('accepts a graph exactly at the budget and rejects one internal edge above it', () => {
|
||||
const packages = 350
|
||||
const packages = 320
|
||||
const edgesAtBudget = Math.floor(MAX_RESOLUTION_WORK_UNITS / packages)
|
||||
|
||||
expect(packages * edgesAtBudget).toBe(MAX_RESOLUTION_WORK_UNITS)
|
||||
@@ -136,8 +136,8 @@ describe('resolution work budget', () => {
|
||||
})
|
||||
|
||||
it('accepts the headroom and rejects a runaway graph with its own measured counts', () => {
|
||||
expect(assertResolutionWorkBudget({ dshPackagesPerVersion: 300, checkedDshEdges: 2800 }))
|
||||
.toBe(840_000)
|
||||
expect(assertResolutionWorkBudget({ dshPackagesPerVersion: 350, checkedDshEdges: 3200 }))
|
||||
.toBe(1_120_000)
|
||||
expect(() => assertResolutionWorkBudget({ dshPackagesPerVersion: 400, checkedDshEdges: 3000 }))
|
||||
.toThrow(`400 package(s) per release x 3000 internal edge(s) = 1200000 unit(s), budget ${String(MAX_RESOLUTION_WORK_UNITS)} unit(s)`)
|
||||
})
|
||||
|
||||
@@ -24,17 +24,19 @@ const DEPENDENCY_FIELDS = ['dependencies', 'optionalDependencies', 'peerDependen
|
||||
* edges of its peer target (`canPlacePeers` calls `checkCanPlaceNoCurrent`,
|
||||
* which walks `depValid` over that target's edges), so resolution time grows
|
||||
* with this product and not with the package count alone. Measured on an idle
|
||||
* M-series host: 277 package(s) x 2524 internal edge(s) = 699,148 unit(s) cost
|
||||
* 180.7 s of npm time out of 185.1 s end to end. The budget adds 25% headroom
|
||||
* M5 Pro with Node 24.17.0 and npm 11.13.0: 320 package(s) x 2836 internal
|
||||
* edge(s) = 907,520 unit(s) cost 71.03–97.73 s of npm time; the 315-package,
|
||||
* 2782-edge graph took 75.27 s on the same host. The budget adds 25% headroom
|
||||
* for ordinary product growth, so a graph that grows here fails with its own
|
||||
* measured counts instead of failing on whichever runner happens to be slow.
|
||||
* Raising it requires re-measuring the resolution and recording the new counts.
|
||||
*/
|
||||
export const MAX_RESOLUTION_WORK_UNITS = 880_950
|
||||
export const MAX_RESOLUTION_WORK_UNITS = 1_134_400
|
||||
|
||||
/**
|
||||
* Measured npm seconds per resolution work unit on the reference host: 180.66 s
|
||||
* for the 699,148 units of the graph the budget was derived from.
|
||||
* for 699,148 units. This conservative hang-guard calibration also covers the
|
||||
* current 907,520-unit graph measured at 71.03–97.73 s on an M5 Pro.
|
||||
*/
|
||||
const SECONDS_PER_WORK_UNIT = 2.6e-4
|
||||
|
||||
|
||||
Reference in New Issue
Block a user