mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-11-09 20:07:43 +08:00
Merge pull request #3451 from deepseek-harness/worktree/fix-model-listing-discovery
fix(llm): expand model listing discovery
This commit is contained in:
Yichen Jiang
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-04-draft-provider-endpoint-interrogation.md
|
||||
2026-08-04-draft-provider-endpoint-interrogation.md: d4112d813ad4f5781b74639209d13952e459f7dd
|
||||
2026-08-04-draft-provider-endpoint-interrogation.zh.md: 1626a34cb3163949d70688cefeec77d328c62caa
|
||||
2026-08-04-draft-provider-endpoint-interrogation.md: a6d8d32f12711744cb1380d2ab2767dcd3dd63f6
|
||||
2026-08-04-draft-provider-endpoint-interrogation.zh.md: e162ddb30706e871a2a42b5bbe7e09a451296289
|
||||
|
||||
+6
-6
@@ -6,7 +6,7 @@ English | [中文](2026-08-04-draft-provider-endpoint-interrogation.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Once a pi-ai route became [a declaration rather than a catalog lookup](2026-08-03-pi-ai-declared-provider-catalog.md), a person adding an OpenAI-compatible gateway had to know its model ids before they could configure it. The adapter no longer constrains them to an installed catalog, which is the point, but it also means nothing tells the user what the endpoint actually serves — and most of these endpoints do publish that list at `GET /models`.
|
||||
Once a pi-ai route became [a declaration rather than a catalog lookup](2026-08-03-pi-ai-declared-provider-catalog.md), a person adding a compatible gateway had to know its model ids before they could configure it. The adapter no longer constrains them to an installed catalog, which is the point, but it also means nothing tells the user what the endpoint actually serves — and OpenAI- and Anthropic-compatible endpoints publish that list through protocol-specific model-listing routes.
|
||||
|
||||
The obvious answer, a dynamic runtime catalog refreshed in the background, was rejected with the layer below it: it makes a route's model list external mutable state needing a cache, an invalidation story, and an offline path, while the product need is narrower. What is needed is a *question asked once*, whose answer the user adopts into `settings.yaml` — so `settings.yaml` remains the only thing deciding what a route serves.
|
||||
|
||||
@@ -21,7 +21,7 @@ Interrogation is keyed by **settings namespace**, not by provider route:
|
||||
- `LlmDiscoveredModel` makes every field but `id` optional, because most listings disclose an id and nothing else. The reply is candidates, not a catalog: a surface adopting one still owes the capacities the adapter requires.
|
||||
- `llm.discoverModels` carries the same draft over the wire. Its `apiKey` is the third and last payload on which a secret may ride, alongside `settings.update`/`mutate` and `credentials.set`, and it is never stored or echoed back. It does ride the client's outgoing envelope like every other secret-bearing payload, where a `subscribeEnvelopes()` observer can see it; redacting that tap is a configuration-plane-wide change, not this method's to make alone. Connection authenticates the method with the complete Host API: it makes the host issue a GET to a caller-chosen URL and reports the outcome, which an anonymous caller must not receive. Every refusal folds into `model-discovery-failed`, whose message is the adapter's own text and whose details name the endpoint asked but never the credential offered.
|
||||
|
||||
`dsh-llm-pi-ai` implements the wire path as a plain `GET {baseURL}/models`, reading `openai-completions` and `openai-responses`: their `GET /models` shape with bearer auth is the one a gateway, a self-hosted server, and the official endpoints all agree on. Profile resolution rejects names and values Fetch cannot represent, so a malformed deployment header is reported as a configuration error before interrogation. Configured profile headers are installed first; the fixed JSON accept header, a typed-or-stored bearer credential, and Harness attribution then win case-insensitive collisions in that order. Azure is excluded despite its OpenAI lineage — it authenticates with an `api-key` header and requires an `api-version` query — and Codex uses OAuth; both would have reported an authentication failure as a provider with no models. Every other protocol answers `DISCOVERY_UNSUPPORTED`, so the surface falls back to hand-entry rather than reporting a guessed response shape as an empty provider. `baseURL` is treated as a prefix rather than a URL to resolve against, so a deployment path such as `https://gateway.example/openai/v1` keeps its segments. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared `content-length` is checked first as a courtesy but never trusted as the bound, matching `dsh-web-fetch`'s two-stage shape for its own caller-supplied URLs.
|
||||
`dsh-llm-pi-ai` applies the protocol-specific listing routes, authentication, URL normalization, response formats, and metadata rules recorded by [protocol-specific model listing discovery](2026-09-02-protocol-specific-model-listing-discovery.md). Profile resolution rejects names and values Fetch cannot represent, so a malformed deployment header is reported as a configuration error before interrogation. Configured profile headers are installed first; fixed protocol headers, a typed-or-stored protocol credential, and Harness attribution then win their case-insensitive collisions. A protocol without a documented listing contract answers `DISCOVERY_UNSUPPORTED`, so the surface falls back to hand-entry rather than reporting guessed response fields as an empty provider. `baseURL` is treated as a prefix rather than a URL to resolve against, so deployment path segments remain intact. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared `content-length` is checked first as a courtesy but never trusted as the bound, matching `dsh-web-fetch`'s two-stage pattern for its own caller-supplied URLs.
|
||||
|
||||
### Why not pi-ai's own refresh machinery
|
||||
|
||||
@@ -35,16 +35,16 @@ pi-ai supplies `createProvider({ fetchModels })` plus `Models.refresh()` and a `
|
||||
|
||||
**Have the host read the entire stored profile instead of accepting a draft.** No secret would cross the wire for an already-configured provider. But adding a provider would then require saving an unusable configuration first, and a form whose endpoint was edited but not yet saved would silently interrogate the old one. The draft remains authoritative for the endpoint and protocol. The narrow Host-side exceptions are the stored credential, which is write-only, and profile headers, which remain deployment configuration rather than Models-page fields.
|
||||
|
||||
**Interrogate every pi-ai protocol.** Anthropic's listing happens to share OpenAI's envelope, and Google's does not. Supporting the ones that are easy would make coverage arbitrary and, worse, make a wrong guess at a response shape indistinguishable from a provider with no models. A protocol that says it cannot be interrogated sends the user to hand-entry, which is the documented fallback.
|
||||
**Interrogate every pi-ai protocol.** Coverage based on convenient response similarities would be arbitrary and would make a wrong guess indistinguishable from a provider with no models. Anthropic is included only through its documented native listing contract, as the [protocol-specific extension](2026-09-02-protocol-specific-model-listing-discovery.md) records; Google's field set and Azure's request contract differ, while Codex uses OAuth. An unsupported protocol sends the user to hand-entry, which remains the documented fallback.
|
||||
|
||||
**Buffer the reply with `response.text()` and check its length.** Simpler, but the bound would arrive after the bytes did, and the endpoint is whatever URL the user typed.
|
||||
|
||||
## Consequences
|
||||
|
||||
A person adding a gateway can ask it what it serves instead of hunting through its documentation, and the answer arrives as candidates they choose from rather than as configuration written behind their back. An already-configured enterprise gateway uses the same deployment headers for interrogation and model requests without adding a header injection field to the browser protocol. The seam gained a registry that is deliberately small: one offer per namespace, no storage, no lifecycle beyond the fiber.
|
||||
A person adding a gateway can ask it what it serves instead of hunting through its documentation, and the answer arrives as candidates they choose from rather than as configuration written behind their back. When an endpoint discloses richer metadata, adopting a candidate fills its id, name, context window, and output-token cap into the editable Web row. Search preserves hidden selections, selecting all adds the visible results, and deselecting all clears every result so a filtered picker cannot submit hidden models accidentally. An already-configured enterprise gateway uses the same deployment headers and Harness `User-Agent` for interrogation and model requests without adding a header injection field to the browser protocol. The seam gained a registry that is deliberately small: one offer per namespace, no storage, no lifecycle beyond the fiber.
|
||||
|
||||
What it costs: the wire gained a third secret-carrying payload, so the configuration plane's write-only surface is now three methods rather than two. Discovery coverage is protocol-shaped rather than provider-shaped — an Anthropic-compatible gateway must be filled in by hand even though its listing would parse. And because nothing re-runs the question, a model list is still only as current as its last edit; that is the same trade the layer below made deliberately.
|
||||
What it costs: the wire gained a third secret-carrying payload, so the configuration plane's write-only surface is now three methods rather than two. Discovery coverage remains protocol-shaped rather than provider-shaped, and an endpoint using an unsupported request contract must be filled in by hand. Because nothing re-runs the question, a model list is still only as current as its last edit; that is the same trade the layer below made deliberately.
|
||||
|
||||
## Testing
|
||||
|
||||
`packages/llm/llm/tests/topology.spec.ts` covers the registry: one offer per namespace, disposal with the fiber, normalization that drops duplicate and unusable ids without inventing capacities, the `NO_DISCOVERY`/`INVALID_DISCOVERY` refusals, and the `model-discovery-failed` Remote mapping. `packages/llm/llm-pi-ai/tests/discovery.spec.ts` drives the probe against local HTTP servers — a listing with and without disclosed capacities, a preserved deployment path, an absent credential, a configured route supplying its stored credential and headers while a typed key wins without resolving the stored one, a catalog route answering without resolving one at all, dropped rows, 401/403 versus a server fault, a non-listing and a non-JSON body, an unreachable endpoint, caller cancellation, an unsupported protocol, and the size ceiling in both its declared-length and streamed forms. `packages/llm/llm-pi-ai/tests/loader-composition.spec.ts` boots settings and credentials through the Loader and proves settings-only headers reach `GET /models` with request-owned headers winning collisions. `packages/llm/llm-pi-ai/tests/adapter.spec.ts` rejects profile headers Fetch cannot represent, and `packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts` proves a settings write reports that configuration error while its last good routes keep serving. `packages/client/connection/tests/node-half.host.spec.ts` pins the `llm/discoverModels` `/api` carrier registration, while `packages/client/ui-settings-models/tests/provider-form.client.spec.tsx` verifies that the draft reaches the Remote whole, absent fields stay absent, and no settings namespace or credential is written before selection.
|
||||
`packages/llm/llm/tests/topology.spec.ts` covers the registry: one offer per namespace, disposal with the fiber, normalization that drops duplicate and unusable ids without inventing capacities, the `NO_DISCOVERY`/`INVALID_DISCOVERY` refusals, and the `model-discovery-failed` Remote mapping. `packages/llm/llm-pi-ai/tests/discovery.spec.ts` drives the probe against local HTTP servers — standard arrays and enriched objects with every accepted metadata spelling, Anthropic's native path, headers, and capacity fields, route keys that differ from nested canonical ids, name fallback, a preserved deployment path, an absent credential, a configured route supplying its stored credential and headers while a typed key wins without resolving the stored one, a catalog route answering without resolving one at all, dropped rows, 401/403 versus a server fault, a non-listing and a non-JSON body, an unreachable endpoint, caller cancellation, an unsupported protocol, and the size ceiling in both its declared-length and streamed forms. `packages/llm/llm-pi-ai/tests/loader-composition.spec.ts` boots settings and credentials through the Loader and proves settings-only headers reach `GET /models` with request-owned headers winning collisions. `packages/llm/llm-pi-ai/tests/adapter.spec.ts` rejects profile headers Fetch cannot represent, and `packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts` proves a settings write reports that configuration error while its last good routes keep serving. `packages/client/connection/tests/node-half.host.spec.ts` pins the `llm/discoverModels` `/api` carrier registration, while the component and built-Web settings tests verify that the complete draft reaches the Remote, absent fields stay absent, selected metadata fills all four editable model fields, tuned rows win over rediscovery, filtered deselection clears hidden candidates, and no settings namespace or credential is written before selection.
|
||||
|
||||
+6
-6
@@ -6,7 +6,7 @@ Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
当 pi-ai 路由变成[一份声明而非 catalog 查表](2026-08-03-pi-ai-declared-provider-catalog.zh.md)之后，要接入一个 OpenAI 兼容网关的人，必须先知道它的模型 id 才能完成配置。适配器不再把人限制在已安装 catalog 里——这正是那次改动的目的——但也意味着没有任何东西告诉用户该端点究竟服务什么，而这类端点大多在 `GET /models` 上公布了这份列表。
|
||||
当 pi-ai 路由变成[一份声明而非 catalog 查表](2026-08-03-pi-ai-declared-provider-catalog.zh.md)之后，要接入一个兼容网关的人，必须先知道它的模型 id 才能完成配置。适配器不再把人限制在已安装 catalog 里——这正是那次改动的目的——但也意味着没有任何东西告诉用户该端点究竟服务什么，而 OpenAI 与 Anthropic 兼容端点会通过各自协议的模型列表路由公布这份信息。
|
||||
|
||||
显而易见的答案——后台刷新的运行时动态 catalog——已随下层一并被拒绝：它会把路由的模型列表变成需要缓存、失效语义与离线路径的外部可变状态，而产品需求要窄得多。真正需要的是*一次性询问*，其答案由用户采纳进 `settings.yaml`，从而让 `settings.yaml` 始终是决定路由服务内容的唯一真源。
|
||||
|
||||
@@ -21,7 +21,7 @@ Status: implemented
|
||||
- `LlmDiscoveredModel` 除 `id` 外每个字段都可选，因为大多数列表只公布 id。回复是候选而非 catalog：采纳其中一条的界面仍要补上适配器所需的容量。
|
||||
- `llm.discoverModels` 把同一份草稿送过协议层。它的 `apiKey` 是可承载机密的第三个、也是最后一个载荷（另两个是 `settings.update`/`mutate` 与 `credentials.set`），且绝不被存储或回显。它确实会像其他承载机密的载荷一样随客户端外发信封同行，`subscribeEnvelopes()` 观察者看得到；把那个抽头脱敏是整个配置面的改动，不该由这一个方法独自决定。Connection 用与完整 Host API 相同的会话认证该方法：它让宿主向调用方选定的 URL 发起 GET 并回报结果，匿名调用者绝不能获得这类探测能力。每一种拒绝都折叠为 `model-discovery-failed`，其消息是适配器自己的文本，details 点名被询问的端点，绝不点名所提供的凭据。
|
||||
|
||||
`dsh-llm-pi-ai` 的实现只是一次朴素的 `GET {baseURL}/models`，且仅限 OpenAI 兼容协议。它们的列表形状是网关、自建服务与官方端点三方一致认可的那一种，而这正是该动作存在的场景。Profile 解析会拒绝 Fetch 无法表示的名称与值，因此格式错误的部署 header 会在询问前以配置错误报告。已配置的 profile headers 最先装入；固定的 JSON accept header、键入或已存的 bearer 凭据以及 Harness attribution 随后依次以大小写不敏感方式赢得冲突。其余协议一律以 `DISCOVERY_UNSUPPORTED` 回答，让界面回退到手工填写，而不是把猜错的响应形状报成一个空提供方。`baseURL` 按前缀而非待解析 URL 处理，因此 `https://gateway.example/openai/v1` 这类部署路径会保留其路径段。回复在四兆字节上限下读取，且上限落在实际收到的字节上——端点是用户自己填的 URL，因此会先看声明的 `content-length` 作为善意提示，但绝不把它当作边界；这与 `dsh-web-fetch` 面对自己的调用方提供 URL 时所用的两段式形状一致。
|
||||
`dsh-llm-pi-ai` 会应用[协议特定模型列表发现](2026-09-02-protocol-specific-model-listing-discovery.zh.md)记录的列表路由、认证、URL 归一化、响应格式与元数据规则。Profile 解析会拒绝 Fetch 无法表示的名称与值，因此格式错误的部署 header 会在询问前以配置错误报告。已配置的 profile headers 最先装入；固定协议 headers、键入或已存的协议凭据以及 Harness attribution 随后分别以大小写不敏感方式赢得相应冲突。没有文档化列表约定的协议会以 `DISCOVERY_UNSUPPORTED` 回答，让界面回退到手工填写，而不是把猜错的响应字段报成一个空提供方。`baseURL` 按前缀而非待解析 URL 处理，因此部署路径段会保持不变。回复在四兆字节上限下读取，且上限落在实际收到的字节上——端点是用户自己填的 URL，因此会先看声明的 `content-length` 作为善意提示，但绝不把它当作边界；这与 `dsh-web-fetch` 面对自己的调用方提供 URL 时所用的两段式模式一致。
|
||||
|
||||
### 为什么不用 pi-ai 自己的 refresh 机制
|
||||
|
||||
@@ -35,16 +35,16 @@ pi-ai 提供了 `createProvider({ fetchModels })` 加上 `Models.refresh()` 与
|
||||
|
||||
**让 Host 读取整个已存 profile，而不是接受草稿。** 对已配置好的提供方来说，不会有机密跨越协议层。但这样一来新增提供方就必须先保存一份不可用的配置，而端点已改却尚未保存的表单会静默地去询问旧地址。草稿仍是端点和协议的权威来源。Host 侧的狭窄例外是只写的已存凭据，以及仍属部署配置、而非 Models 页面字段的 profile headers。
|
||||
|
||||
**询问 pi-ai 的每一种协议。** Anthropic 的列表恰好与 OpenAI 共用同一层信封，而 Google 的不是。只支持容易的那几种会让覆盖范围变得任意；更糟的是，猜错的响应形状会与「该提供方没有模型」无法区分。一个明说自己无法被询问的协议，会把用户送去手工填写——那正是既定的回退路径。
|
||||
**询问每一种 pi-ai 协议。** 根据便利的响应相似性选择覆盖范围会显得武断，也会让猜错的响应与「提供方没有模型」无法区分。Anthropic 仅通过其文档化原生列表约定纳入支持，具体由[协议特定扩展](2026-09-02-protocol-specific-model-listing-discovery.zh.md)记录；Google 的字段集合与 Azure 的请求约定不同，Codex 则使用 OAuth。不支持的协议会把用户送去手工填写，这仍是既定回退路径。
|
||||
|
||||
**用 `response.text()` 缓冲整个回复再判断长度。** 更简单，但上限会在字节已经到达之后才生效，而端点是用户随手填的任意 URL。
|
||||
|
||||
## Consequences
|
||||
|
||||
接入网关的人可以直接问它服务什么，而不必去翻它的文档；答案以候选形式抵达，由用户自己挑选，而不是被背着写进配置。已配置的企业网关会为询问与模型请求使用同一组部署 headers，而无需给浏览器协议增加 header 注入字段。seam 因此多了一个刻意保持很小的注册表：每个 namespace 一份、不存储、生命周期不超出 fiber。
|
||||
接入网关的人可以直接问它服务什么，而不必去翻它的文档；答案以候选形式抵达，由用户自己挑选，而不是被背着写进配置。当端点公布了更丰富的元数据时，采纳候选会把 id、名称、上下文窗口与最大输出 token 数填进 Web 的可编辑行。搜索会保留隐藏项的勾选状态，全选会加入可见结果，而取消全选会清空全部结果，因此筛选后的选择器不会意外提交隐藏模型。已配置的企业网关会为询问与模型请求使用同一组部署 headers 和 Harness `User-Agent`，而无需给浏览器协议增加 header 注入字段。seam 因此多了一个刻意保持很小的注册表：每个 namespace 一份、不存储、生命周期不超出 fiber。
|
||||
|
||||
代价是：协议层多了第三个承载机密的载荷，配置面的只写接口从两个方法变成三个。发现覆盖范围按协议而非按提供方划分——一个 Anthropic 兼容网关即便其列表能被解析，也仍须手工填写。而且由于没有任何环节会重跑该询问，模型列表的新鲜度依旧只到最近一次编辑为止；这与下层刻意做出的取舍是同一个。
|
||||
代价是：协议层多了第三个承载机密的载荷，配置面的只写接口从两个方法变成三个。发现覆盖范围仍按协议而非按提供方划分，使用不受支持请求约定的端点仍须手工填写。由于没有任何环节会重跑该询问，模型列表的新鲜度依旧只到最近一次编辑为止；这与下层刻意做出的取舍是同一个。
|
||||
|
||||
## Testing
|
||||
|
||||
`packages/llm/llm/tests/topology.spec.ts` 覆盖注册表：每个 namespace 一份、随 fiber dispose（资源释放）、丢弃重复与不可用 id 且不凭空补容量的归一化、`NO_DISCOVERY`/`INVALID_DISCOVERY` 两种拒绝，以及 `model-discovery-failed` Remote 映射。`packages/llm/llm-pi-ai/tests/discovery.spec.ts` 针对本地 HTTP 服务器驱动探测——含与不含公布容量的列表、被保留的部署路径、无凭据、已配置路由提供自己的已存凭据与 headers 且键入的密钥无需解析已存凭据便可压过它、catalog 路由完全不解析凭据即作答、被丢弃的行、401/403 与服务器故障之别、非列表与非 JSON 响应、不可达端点、调用方取消、不支持的协议，以及尺寸上限的「声明长度」与「流式」两种形态。`packages/llm/llm-pi-ai/tests/loader-composition.spec.ts` 通过 Loader 启动 settings 与 credentials，并证明仅配置在 settings 中的 headers 会抵达 `GET /models`，且请求所持有的 headers 赢得冲突。`packages/llm/llm-pi-ai/tests/adapter.spec.ts` 拒绝 Fetch 无法表示的 profile headers，`packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts` 证明 settings 写入会报告该配置错误，同时上一组可用路由仍继续服务。`packages/client/connection/tests/node-half.host.spec.ts` 固定 `llm/discoverModels` 的 `/api` 承载注册，`packages/client/ui-settings-models/tests/provider-form.client.spec.tsx` 则验证草稿完整抵达 Remote、缺席字段保持缺席，以及选择前没有 settings namespace 或凭据被写入。
|
||||
`packages/llm/llm/tests/topology.spec.ts` 覆盖注册表：每个 namespace 一份、随 fiber dispose（资源释放）、丢弃重复与不可用 id 且不凭空补容量的归一化、`NO_DISCOVERY`/`INVALID_DISCOVERY` 两种拒绝，以及 `model-discovery-failed` Remote 映射。`packages/llm/llm-pi-ai/tests/discovery.spec.ts` 针对本地 HTTP 服务器驱动探测——包括采用每种受支持元数据拼写的标准数组与富信息对象、Anthropic 原生路径、headers 与容量字段、不同于嵌套规范 id 的路由键、名称回退、被保留的部署路径、无凭据、已配置路由提供自己的已存凭据与 headers 且键入的密钥无需解析已存凭据便可压过它、catalog 路由完全不解析凭据即作答、被丢弃的行、401/403 与服务器故障之别、非列表与非 JSON 响应、不可达端点、调用方取消、不支持的协议，以及尺寸上限的「声明长度」与「流式」两种形态。`packages/llm/llm-pi-ai/tests/loader-composition.spec.ts` 通过 Loader 启动 settings 与 credentials，并证明仅配置在 settings 中的 headers 会抵达 `GET /models`，且请求所持有的 headers 赢得冲突。`packages/llm/llm-pi-ai/tests/adapter.spec.ts` 拒绝 Fetch 无法表示的 profile headers，`packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts` 证明 settings 写入会报告该配置错误，同时上一组可用路由仍继续服务。`packages/client/connection/tests/node-half.host.spec.ts` 固定 `llm/discoverModels` 的 `/api` 承载注册，而设置页的组件测试和构建后 Web 测试则验证完整草稿抵达 Remote、缺席字段保持缺席、所选元数据填满四个可编辑模型字段、用户调整过的行优先于重新发现结果、筛选后的取消选择会清除隐藏候选项，以及选择前没有 settings namespace 或凭据被写入。
|
||||
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-02-protocol-specific-model-listing-discovery.md
|
||||
2026-09-02-protocol-specific-model-listing-discovery.md: 69e684c284ff57da7ed61fd5dc417fd110d3e5b7
|
||||
2026-09-02-protocol-specific-model-listing-discovery.zh.md: 682806fb72c2cc24572e6e351d4d388cead84dc0
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# Agent Note: Reading protocol-specific model listings
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-09-02-protocol-specific-model-listing-discovery.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The [draft provider interrogation](2026-08-04-draft-provider-endpoint-interrogation.md) originally read the OpenAI-compatible `data` array only. Some compatible gateways instead publish an enriched `models` object, while Anthropic publishes a native model-listing route with different authentication and URL rules. Treating either case as unsupported forced a user to copy model ids and capacities by hand even though the endpoint disclosed them.
|
||||
|
||||
One gateway could be made to return an OpenAI-style array by sending an OpenAI SDK `User-Agent`. That behavior was undocumented, changed request attribution, and made the reply depend on a client identity rather than on a supported response parser.
|
||||
|
||||
## Decision
|
||||
|
||||
`dsh-llm-pi-ai` reads model listings according to the selected protocol. `openai-completions` and `openai-responses` use `GET {baseURL}/models` with bearer authentication. `anthropic-messages` uses `GET /v1/models?limit=1000` with `x-api-key` and `anthropic-version: 2023-06-01`. The Anthropic page size is the documented maximum; discovery does not follow `has_more`, so an endpoint advertising more than 1,000 models exposes only its first page.
|
||||
|
||||
Anthropic SDK resource methods append `/v1` themselves, and gateway documentation publishes the API root both with and without that suffix. The listing URL therefore treats a drafted Anthropic `baseURL` ending in `/v1` as the same API root as the address without it, and it is the only place that segment is normalized: model requests receive the configured `baseURL` unchanged, exactly as pi-ai handles it. Deployment path prefixes remain intact: `https://gateway.example/tenant/v1` and `https://gateway.example/tenant` both list at `/tenant/v1/models`.
|
||||
|
||||
The parser accepts a `data` array or an enriched `models` object, with a present array taking precedence. Array entries use their `id`; object entries use the property key because a nested `id` may name a canonical model instead of the route alias accepted on requests. Only object-valued map entries are considered models, so primitive directory metadata cannot become a candidate accidentally. A nested `id` is the fallback for an empty property key.
|
||||
|
||||
The parser normalizes the supported name and capacity spellings into `LlmDiscoveredModel`. A missing display name becomes the request id so adoption fills a complete editable row. The request keeps the Harness attribution headers; response parsing, not client impersonation, provides gateway compatibility.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Follow every Anthropic page.** Cursor traversal would return listings larger than 1,000 entries, but it adds multi-request failure, cancellation, cursor-progress, and aggregate-size behavior to a configuration action. The implementation requests Anthropic's maximum page and documents the remaining truncation.
|
||||
|
||||
**Normalize the inference base as well.** Stripping the same `/v1` segment before model routing would let a `/v1` address both list and serve, but it moves request URL rules out of pi-ai and into this package for one protocol. Model requests keep pi-ai's own handling of `baseURL`; the listing request is the only URL this package builds.
|
||||
|
||||
**Refuse a trailing `/v1`.** A load-time or discovery-time rejection would name the mistake early, but gateway documentation publishes the `/v1` spelling, so a user pasting a documented address would be turned away from a listing that works.
|
||||
|
||||
**Send an OpenAI SDK `User-Agent` for discovery.** This made one gateway return `data`, but it misattributed Harness traffic and relied on an undocumented client-name branch. Reading both known reply formats keeps attribution accurate.
|
||||
|
||||
**Adopt every property of a `models` object.** A primitive-valued property does not prove that its key is a model id and may be directory metadata such as a count or status. Restricting entries to records avoids inventing model candidates.
|
||||
|
||||
## Consequences
|
||||
|
||||
The Models page can interrogate OpenAI-compatible gateways and Anthropic Messages endpoints without changing request identity. Discovered candidates carry route ids, names, context windows, and output-token caps when the endpoint provides them, and name-only listings still receive an editable label through the id fallback. Anthropic addresses list in either root or `/v1` form; model requests use the configured address as pi-ai receives it.
|
||||
|
||||
The supported formats remain an explicit compatibility set rather than arbitrary JSON inference. Anthropic accounts with more than 1,000 visible models require hand-entry for entries outside the first page, and primitive-valued `models` properties are ignored.
|
||||
|
||||
## Testing
|
||||
|
||||
Local HTTP-server tests pin both accepted response formats, field normalization, name fallback, ignored malformed entries, Anthropic headers, the maximum-page query, and both spellings of the Anthropic root. Replies recorded from OpenRouter, models.dev, and DeepSeek on 2026-09-02, together with the example reply in Anthropic's List Models reference, live under `packages/llm/llm-pi-ai/tests/fixtures/model-listings/` and replay through the parser, so the accepted field spellings are pinned to real endpoints and the published reference rather than to hand-written samples.
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# Agent Note: 读取协议特定的模型列表
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-09-02-protocol-specific-model-listing-discovery.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
[提供方草稿询问决策](2026-08-04-draft-provider-endpoint-interrogation.zh.md)最初只读取 OpenAI 兼容的 `data` 数组。一些兼容网关改为公布富信息 `models` 对象，而 Anthropic 公布了具有不同认证与 URL 规则的原生模型列表路由。把任一情况视为不受支持，都会迫使用户手工复制模型 id 和容量，即使端点已经公布这些信息。
|
||||
|
||||
向一个网关发送 OpenAI SDK `User-Agent` 可以使其返回 OpenAI 风格数组。该行为没有文档，改变了请求归属，并使回答取决于客户端身份而非受支持的响应解析器。
|
||||
|
||||
## 决策
|
||||
|
||||
`dsh-llm-pi-ai` 按所选协议格式读取模型列表。`openai-completions` 与 `openai-responses` 以 bearer 认证使用 `GET {baseURL}/models`。`anthropic-messages` 以 `x-api-key` 和 `anthropic-version: 2023-06-01` 使用 `GET /v1/models?limit=1000`。Anthropic 页大小采用文档规定的最大值；模型发现不会继续跟随 `has_more`，因此公布超过 1,000 个模型的端点只会暴露第一页。
|
||||
|
||||
Anthropic SDK 资源方法会自行追加 `/v1`，而网关文档会同时发布带与不带该后缀的 API 根地址。因此列表 URL 会把末尾为 `/v1` 的 Anthropic `baseURL` 草稿视为与不带该后缀的地址相同的 API 根地址，并且只有它会归一化这一段：模型请求收到的仍是配置原样的 `baseURL`，与 pi-ai 的处理完全一致。部署路径前缀会保留：`https://gateway.example/tenant/v1` 与 `https://gateway.example/tenant` 都在 `/tenant/v1/models` 列表。
|
||||
|
||||
解析器接受 `data` 数组或富信息 `models` 对象，并在数组存在时优先使用它。数组条目使用自身的 `id`；对象条目使用属性键，因为嵌套 `id` 可能指向规范模型，而不是请求所接受的路由别名。只有值为对象的映射条目才视为模型，因此原始类型的目录元数据不会意外成为候选。空属性键才会回退到嵌套 `id`。
|
||||
|
||||
解析器会把受支持的名称与容量拼写归一化为 `LlmDiscoveredModel`。缺失的显示名会变成请求 id，使采纳操作填入完整的可编辑行。请求保留 Harness 归属标头；网关兼容性由响应解析提供，而非冒充客户端身份。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**跟随 Anthropic 的所有页面。** 游标遍历可以返回超过 1,000 个条目的列表，但会给配置操作增加多请求失败、取消、游标推进与总大小处理。实现请求 Anthropic 的最大页面，并记录剩余截断限制。
|
||||
|
||||
**同时归一化推理地址。** 在模型路由前截掉同一段 `/v1` 可以让 `/v1` 地址既能列表也能服务，但这会把请求 URL 规则从 pi-ai 挪进本包，且只为一种协议。模型请求保持 pi-ai 自身对 `baseURL` 的处理；列表请求是本包构造的唯一 URL。
|
||||
|
||||
**拒绝末尾的 `/v1`。** 在加载或探测时拒绝可以尽早点出错误，但网关文档发布的就是 `/v1` 写法，照文档粘贴地址的用户会被一个本能工作的列表拒之门外。
|
||||
|
||||
**为模型发现发送 OpenAI SDK `User-Agent`。** 这会让一个网关返回 `data`，但会错误标记 Harness 流量，并依赖未记录的客户端名称分支。读取两种已知响应格式可以保持归属准确。
|
||||
|
||||
**采纳 `models` 对象的每个属性。** 原始类型属性不能证明其键是模型 id，也可能是数量或状态等目录元数据。把条目限制为记录可避免虚构模型候选。
|
||||
|
||||
## 后果
|
||||
|
||||
Models 页面无需改变请求身份，即可询问 OpenAI 兼容网关与 Anthropic Messages 端点。发现的候选会在端点提供时携带路由 id、名称、上下文窗口与最大输出 token 数，只有 id 的列表也会通过 id 回退获得可编辑标签。Anthropic 地址以根地址或 `/v1` 形式都能列表；模型请求使用 pi-ai 收到的配置地址。
|
||||
|
||||
受支持格式仍是显式兼容集合，而不是任意 JSON 推断。可见模型超过 1,000 个的 Anthropic 账户需要手工录入第一页之外的条目，原始类型的 `models` 属性会被忽略。
|
||||
|
||||
## 测试
|
||||
|
||||
本地 HTTP 服务器测试钉住两种受支持响应格式、字段归一化、名称回退、忽略畸形条目、Anthropic 标头、最大页查询，以及 Anthropic 根地址的两种写法。2026-09-02 从 OpenRouter、models.dev 与 DeepSeek 录得的回复，连同 Anthropic List Models 参考文档给出的示例回复，存放在 `packages/llm/llm-pi-ai/tests/fixtures/model-listings/` 下并经解析器回放，因此受支持的字段拼写钉在真实端点与公开参考文档上，而不是手写样例上。
|
||||
@@ -179,7 +179,7 @@ describe('web e2e: Models settings page configures a dormant provider', () => {
|
||||
expect(tripwire.pageErrors).toEqual([])
|
||||
}, 60_000)
|
||||
|
||||
it('filters the discovered model catalog and preserves hidden selections', async () => {
|
||||
it('filters the discovered model catalog and clears hidden selections', async () => {
|
||||
onTestFailed(() => saveFailureShot(page, 'web-e2e-models-picker'))
|
||||
const settingsDialog = page.getByRole('dialog', { name: '设置' })
|
||||
await settingsDialog.getByRole('button', { name: '编辑 minimax-cn' }).click()
|
||||
@@ -206,8 +206,7 @@ describe('web e2e: Models settings page configures a dormant provider', () => {
|
||||
await search.fill('')
|
||||
await expect.poll(async () => boxes.count()).toBe(count)
|
||||
const restored = await boxes.evaluateAll(nodes => nodes.map(node => (node as HTMLInputElement).checked))
|
||||
expect(restored.filter(Boolean)).toHaveLength(count - 1)
|
||||
expect(restored.filter(checked => !checked)).toHaveLength(1)
|
||||
expect(restored).toEqual(Array.from({ length: count }, () => false))
|
||||
await picker.getByRole('button', { name: '全选' }).waitFor()
|
||||
await picker.getByRole('button', { name: '全选' }).click()
|
||||
const snapshot = await captureStableAria(
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/client/ui-settings-models/README.md
|
||||
README.md: b3f07e1a788b223f0f49e37b9a4a54662acf6445
|
||||
README.zh.md: c23dcf95f6b9e9d47b390c3243bf3d8bb0ef26f1
|
||||
README.md: cac2785886346a0c1e81009404fdb0da238bbe40
|
||||
README.zh.md: 7df96a9a1b9474b5e88d8f9814f353f92eb1a31d
|
||||
|
||||
@@ -37,7 +37,7 @@ The collapsed 自定义设置 fold carries the curated extras: `baseURL` for bot
|
||||
|
||||
### Adding and deleting providers
|
||||
|
||||
The add flow is a card carrying the dormant-directory provider select — a bare-mounted `llm-pi-ai` offers its whole installed catalog before any route exists. **Add a custom provider** declares a route pi-ai does not ship; the create card asks for a unique **Provider ID**, an endpoint, a protocol, and at least one uniquely-identified model, because nothing can default those. **Fetch available models** asks the `llm/discoverModels` Remote about the endpoint the form shows, so adding a provider is one pass instead of save-then-return; the reply opens a searchable picker rather than being written, and nothing is written until **Add selected**. Search matches model ids and optional display names without clearing hidden selections, while **Select all** and **Deselect all** affect only the visible results. A row is deletable only when the user layer alone carries it (removal restores the composition base), and its confirmation dialog names the provider.
|
||||
The add flow is a card carrying the dormant-directory provider select — a bare-mounted `llm-pi-ai` offers its whole installed catalog before any route exists. **Add a custom provider** declares a route pi-ai does not ship; the create card asks for a unique **Provider ID**, an endpoint, a protocol, and at least one uniquely-identified model, because nothing can default those. **Fetch available models** asks the `llm/discoverModels` Remote about the endpoint the form shows, so adding a provider is one pass instead of save-then-return; the reply opens a searchable picker rather than being written, and nothing is written until **Add selected**. Each selected candidate copies its id, display name, context window, and output-token cap into the editable row when disclosed, while an existing row retains its user-tuned values. Search matches model ids and optional display names without clearing hidden selections. **Select all** adds the visible results, while **Deselect all** clears the entire selection so hidden results cannot be adopted accidentally. A row is deletable only when the user layer alone carries it (removal restores the composition base), and its confirmation dialog names the provider.
|
||||
|
||||
### First-run dialogs
|
||||
|
||||
@@ -105,7 +105,7 @@ These limits define the editor's field coverage and the page's reach; they are c
|
||||
- **Only the API key and curated fold fields are editable on the card** — the hand-written editor traded schema-generic field coverage for the mockup layout. Retry policy, timeouts, DeepSeek model descriptions, and other advanced fields remain in `settings.yaml`; existing model fields the editor does not show are preserved.
|
||||
- **Credential cleanup is intentionally narrow** — deleting a row removes the configured, writable credential only when its reference is the exact `<ROUTE>_API_KEY` target this page derives. Custom references, environment credentials, and unidentifiable targets are retained because the row cannot prove ownership of them.
|
||||
- **Only pi-ai routes can be hand-declared** — the custom-provider card writes into `llm-pi-ai`, the one namespace whose profiles describe a whole provider. A `llm-deepseek` route is a composition fact, not something this page can create.
|
||||
- **Interrogation covers OpenAI-compatible endpoints** — the adapter reads only that model-list response format, so a gateway speaking another protocol reports that it cannot be asked and its models are entered by hand.
|
||||
- **Interrogation covers OpenAI-compatible and Anthropic Messages endpoints** — OpenAI protocols accept a standard `data` array or an enriched `models` map, while Anthropic uses its native model-listing route; every other protocol reports that it cannot be asked and its models are entered by hand.
|
||||
- **Undeclared live routes render nowhere** — a route registered without a configurable-provider declaration has no settings address; it stays visible in pickers but not on this page's rows.
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -37,7 +37,7 @@ kind: "package-reference"
|
||||
|
||||
### 新增与删除提供方
|
||||
|
||||
「新增」流程是一张承载休眠目录提供方选择框的卡片——裸挂载的 `llm-pi-ai` 在任何路由存在之前就能提供其完整的已安装 catalog。**添加自定义提供方**声明一条 pi-ai 不提供的路由；创建卡片会索要唯一的 **Provider ID**、端点、协议与至少一个可唯一识别的模型，因为没有东西能为它们兜底。**获取可用模型**通过 `llm/discoverModels` Remote 查询表单显示的端点，因此新增提供方一次即可完成，而非先保存再返回；回复打开的是可搜索选择器而非直接写入，只有点击**添加所选**才会写入。搜索会匹配模型 id 与可选显示名称，且不会清除隐藏项的勾选状态；**全选**与**取消全选**只影响可见结果。只有用户层单独携带某行时，该行才可删除（删除会恢复组合基线），其确认对话框会指名该提供方。
|
||||
「新增」流程是一张承载休眠目录提供方选择框的卡片——裸挂载的 `llm-pi-ai` 在任何路由存在之前就能提供其完整的已安装 catalog。**添加自定义提供方**声明一条 pi-ai 不提供的路由；创建卡片会索要唯一的 **Provider ID**、端点、协议与至少一个可唯一识别的模型，因为没有东西能为它们兜底。**获取可用模型**通过 `llm/discoverModels` Remote 查询表单显示的端点，因此新增提供方一次即可完成，而非先保存再返回；回复打开的是可搜索选择器而非直接写入，只有点击**添加所选**才会写入。每个选中候选会在提供方公布相应信息时，把 id、显示名、上下文窗口与最大输出 token 数复制进可编辑行；已经存在的行保留用户调整过的值。搜索会匹配模型 id 与可选显示名称，且不会清除隐藏项的勾选状态。**全选**会加入可见结果，而**取消全选**会清空全部勾选，以免意外采用隐藏结果。只有用户层单独携带某行时，该行才可删除（删除会恢复组合基线），其确认对话框会指名该提供方。
|
||||
|
||||
### 首次运行弹窗
|
||||
|
||||
@@ -105,7 +105,7 @@ kind: "package-reference"
|
||||
- **卡片上只有 API 密钥与精选折叠字段可编辑**：手写编辑器以 schema 通用字段覆盖换取了 mockup 布局。重试策略、超时、DeepSeek 模型说明及其他进阶字段仍留在 `settings.yaml` 中；编辑器未展示的现有模型字段会予以保留。
|
||||
- **凭据清理范围刻意保持狭窄**：删除一行时，仅当其引用与页面派生的 `<ROUTE>_API_KEY` 目标完全一致，才会清除已配置且可写的凭据。自定义引用、环境凭据与无法识别的目标会保留，因为该行无法证明自己拥有它们。
|
||||
- **只有 pi-ai 路由可以手工声明**：自定义提供方卡片写入 `llm-pi-ai`——唯一一个其 profile 描述整个提供方的 namespace。`llm-deepseek` 路由是组合面的事实，不是本页能创建的东西。
|
||||
- **询问只覆盖 OpenAI 兼容端点**：适配器只读这种模型列表响应格式，因此讲其他协议的网关会报告自己无法被询问，其模型需手工填写。
|
||||
- **询问覆盖 OpenAI 兼容与 Anthropic Messages 端点**：OpenAI 协议接受标准 `data` 数组或富信息 `models` 对象，Anthropic 则使用原生模型列表路由；其余协议会报告自己无法被询问，其模型需手工填写。
|
||||
- **未声明的存活路由无处渲染**：未附带可配置提供方声明即注册的路由没有 settings 地址；它在各选择器中仍然可见，但不会出现在本页的行里。
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -297,12 +297,11 @@ export function ModelListEditor(props: ModelListEditorProps): ReactNode {
|
||||
|
||||
const toggleVisibleCandidates = (): void => {
|
||||
setPicked((current) => {
|
||||
const next = new Set(current)
|
||||
if (visibleCandidates.every(candidate => current.has(candidate.id))) {
|
||||
for (const candidate of visibleCandidates) next.delete(candidate.id)
|
||||
} else {
|
||||
for (const candidate of visibleCandidates) next.add(candidate.id)
|
||||
return new Set()
|
||||
}
|
||||
const next = new Set(current)
|
||||
for (const candidate of visibleCandidates) next.add(candidate.id)
|
||||
return next
|
||||
})
|
||||
}
|
||||
|
||||
@@ -529,7 +529,8 @@ describe('endpoint interrogation', () => {
|
||||
|
||||
it('adopts only the picked candidates, keeping a row the user already tuned', async () => {
|
||||
const discover = vi.fn(() => Promise.resolve(ok([
|
||||
{ id: 'kept', contextWindow: 999 }, { id: 'fresh', contextWindow: 4096, name: 'Fresh' },
|
||||
{ id: 'kept', contextWindow: 999 },
|
||||
{ id: 'fresh', contextWindow: 4096, maxTokens: 2048, name: 'Fresh' },
|
||||
])))
|
||||
const { mutate } = await mountSection({
|
||||
discover,
|
||||
@@ -544,11 +545,17 @@ describe('endpoint interrogation', () => {
|
||||
expect(boxes.map(box => box.checked)).toEqual([false, true])
|
||||
fireEvent.click(screen.getByText(en.fetchAdopt))
|
||||
|
||||
expect(screen.getByLabelText<HTMLInputElement>(`${en.modelId} 2`).value).toBe('fresh')
|
||||
expect(screen.getByLabelText<HTMLInputElement>(`${en.modelName} 2`).value).toBe('Fresh')
|
||||
expandModel(2)
|
||||
expect(screen.getByLabelText<HTMLInputElement>(`${en.modelContextWindow} 2`).value).toBe('4096')
|
||||
expect(screen.getByLabelText<HTMLInputElement>(`${en.modelMaxTokens} 2`).value).toBe('2048')
|
||||
|
||||
fireEvent.click(screen.getByText(en.apply))
|
||||
await waitFor(() => { expect(mutate).toHaveBeenCalled() })
|
||||
expect(firstMutate(mutate).ops[0]?.value).toEqual([
|
||||
{ id: 'kept', contextWindow: 111 },
|
||||
{ id: 'fresh', contextWindow: 4096, name: 'Fresh' },
|
||||
{ id: 'fresh', contextWindow: 4096, maxTokens: 2048, name: 'Fresh' },
|
||||
])
|
||||
})
|
||||
|
||||
@@ -661,7 +668,7 @@ describe('endpoint interrogation', () => {
|
||||
expect(firstMutate(mutate).ops[0]?.value).toEqual([{ id: 'a' }, { id: 'b', maxTokens: 2048 }])
|
||||
})
|
||||
|
||||
it('filters by model id or name and scopes bulk selection to visible candidates', async () => {
|
||||
it('filters by model id or name, selects visible candidates, and clears every selection', async () => {
|
||||
const discover = vi.fn(() => Promise.resolve(ok([
|
||||
{ id: 'alpha' }, { id: 'opaque-id', name: 'Beta Display' }, { id: 'gamma' },
|
||||
])))
|
||||
@@ -687,14 +694,17 @@ describe('endpoint interrogation', () => {
|
||||
expect([...dialog.querySelectorAll<HTMLInputElement>('input[type="checkbox"]')]
|
||||
.map(box => box.checked)).toEqual([false])
|
||||
|
||||
// Clearing the filter restores every row and preserves hidden selections.
|
||||
// Deselecting a filtered result must also clear hidden selections so they
|
||||
// cannot be adopted accidentally.
|
||||
fireEvent.change(search, { target: { value: '' } })
|
||||
const boxes = [...dialog.querySelectorAll<HTMLInputElement>('input[type="checkbox"]')]
|
||||
expect(boxes.map(box => box.checked)).toEqual([true, false, true])
|
||||
expect(boxes.map(box => box.checked)).toEqual([false, false, false])
|
||||
|
||||
// Selecting while filtered adds only visible candidates.
|
||||
fireEvent.change(search, { target: { value: 'alpha' } })
|
||||
fireEvent.click(within_(dialog, en.fetchSelectAll))
|
||||
expect(boxes.map(box => box.checked)).toEqual([true, true, true])
|
||||
expect(within_(dialog, en.fetchDeselectAll)).toBeTruthy()
|
||||
fireEvent.change(search, { target: { value: '' } })
|
||||
expect(boxes.map(box => box.checked)).toEqual([true, false, false])
|
||||
|
||||
fireEvent.change(search, { target: { value: 'missing' } })
|
||||
expect(screen.getByText(en.fetchNoMatches)).toBeTruthy()
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: 5994a72f28b0a52890cb7bf7a5bc2ee33eedf418
|
||||
README.zh.md: 5f1c893128714caf24941943c57eaf3ab43314e0
|
||||
README.md: c0aacbe3e0b727cd7b6b163814dc96859b954b9c
|
||||
README.zh.md: 7b8cef7db2ba5cebe3e42345b308716ae0099c03
|
||||
|
||||
@@ -106,7 +106,7 @@ Profiles are re-read once per operation through the optional settings seam: the
|
||||
|
||||
### Discover models from endpoints
|
||||
|
||||
The plugin answers "which models can this provider serve?" for a route a configuration surface is editing or drafting. A route the installed catalog ships is answered from that catalog with no network call; only a route the catalog does not describe is interrogated over the wire (`openai-completions` and `openai-responses` shapes). A named configured route supplies its stored credential and profile `headers` inside the Host, so deployment headers configured through `settings.yaml` or Cordis config reach `GET /models` without becoming discovery-request or Models-page fields; a key typed into the form still wins over the stored credential. The reply is candidate metadata a surface may offer for adoption — nothing is stored, and `settings.yaml` remains the only thing that decides what a route serves.
|
||||
The plugin answers "which models can this provider serve?" for a route a configuration surface is editing or drafting. A route the installed catalog ships is answered from that catalog with no network call; only a route the catalog does not describe is interrogated over the wire. `openai-completions` and `openai-responses` use `GET {baseURL}/models` with bearer auth, while `anthropic-messages` uses native `GET /v1/models?limit=1000` semantics with `x-api-key` and `anthropic-version`; its listing URL accepts the API root with or without a trailing `/v1` because gateway documentation publishes both spellings, and only that listing URL normalizes the segment, so model requests receive the configured `baseURL` unchanged. A named configured route supplies its stored credential and profile `headers` inside the Host, so deployment headers configured through `settings.yaml` or Cordis config reach model discovery without becoming discovery-request or Models-page fields; a key typed into the form still wins over the stored credential. The parser accepts either the standard `data` array or an enriched `models` map, normalizing each candidate's id, display name, context window, and output-token cap; Anthropic's `max_input_tokens` and `max_tokens` feed the same capacity fields, a map key remains the request id even when its entry names a different canonical id, primitive-valued map properties are ignored, and a missing display name falls back to that request id. The reply is candidate metadata a surface may offer for adoption — nothing is stored, and `settings.yaml` remains the only thing that decides what a route serves.
|
||||
|
||||
### Failures and recovery
|
||||
|
||||
@@ -212,6 +212,7 @@ These limits define where the adapter stops and future work begins. They are cur
|
||||
- **The layered merge has no delete for dict keys** — a `reasoningEfforts` level, `modelOverrides` entry, or `compat` field the base declares can be overridden but not removed by the user layer.
|
||||
- **`headers` can carry a credential the redactor never sees** — profile resolution rejects names and values Fetch cannot represent, but the dict remains plain strings; store credentials as `apiKeyEnv` references.
|
||||
- **A route's catalog never refreshes itself** — the catalog is whatever `settings.yaml` says; nothing here queries a provider for the models it serves.
|
||||
- **Anthropic discovery reads at most 1,000 models** — the request uses the API's maximum page size but does not traverse `has_more`; entries beyond the first page must be added by hand.
|
||||
- **One wire protocol per route** — a mixed-protocol catalog route cannot host a model of the other protocol; splitting the provider across two route keys is the workaround.
|
||||
- **A modality declaration is not verified** — a model declaring `image` its gateway does not serve is refused by the provider after prompt admission. The durable image remains in history and the same misdeclared model can fail again; switching to a text-only model remains possible because the shared LLM runtime projects image references into stable text for that request.
|
||||
- **An unauthenticated route depends on its protocol** — a route naming no credential resolves as configured-but-keyless, but pi-ai's OpenAI-compatible implementation still requires an API key or an `Authorization` header, so a keyless local server needs a placeholder credential referenced by `apiKeyEnv` or an `Authorization` entry in `headers`.
|
||||
|
||||
@@ -106,7 +106,7 @@ profile 通过可选 settings seam 每次操作重新读取：base 与用户的
|
||||
|
||||
### 从端点发现模型
|
||||
|
||||
插件会回答"该提供方可以提供哪些模型？"，供配置界面正在编辑或起草的路由使用。已安装目录提供的路由直接由目录回答，不发网络请求；只有目录未描述的路由才会经网络询问（`openai-completions` 与 `openai-responses` 形状）。已配置且具名的路由会在 Host 内部提供已存凭据与 profile `headers`，因此通过 `settings.yaml` 或 Cordis 配置设置的部署标头可以到达 `GET /models`，但不会成为发现请求或 Models 页面的字段；表单中新键入的密钥仍优先于已存凭据。回答是界面可以提供给用户采纳的候选元数据——不存储任何内容，`settings.yaml` 仍然是决定路由服务内容的唯一事实。
|
||||
插件会回答"该提供方可以提供哪些模型？"，供配置界面正在编辑或起草的路由使用。已安装目录提供的路由直接由目录回答，不发网络请求；只有目录未描述的路由才会经网络询问。`openai-completions` 与 `openai-responses` 使用带 bearer 鉴权的 `GET {baseURL}/models`，`anthropic-messages` 则以 `x-api-key` 和 `anthropic-version` 使用原生 `GET /v1/models?limit=1000` 语义；其列表 URL 接受带或不带末尾 `/v1` 的 API 根地址，因为网关文档两种写法都会发布，且只有该列表 URL 会归一化这一段，模型请求收到的仍是配置原样的 `baseURL`。已配置且具名的路由会在 Host 内部提供已存凭据与 profile `headers`，因此通过 `settings.yaml` 或 Cordis 配置设置的部署标头可以到达模型发现请求，但不会成为发现请求或 Models 页面的字段；表单中新键入的密钥仍优先于已存凭据。解析器接受标准 `data` 数组或富信息 `models` 对象，并归一化每个候选的 id、显示名、上下文窗口与最大输出 token 数；Anthropic 的 `max_input_tokens` 与 `max_tokens` 会进入相同容量字段，即使对象条目点名了另一个规范 id，对象键仍是请求 id，原始类型的对象属性会被忽略，缺失的显示名则回退到该请求 id。回答是界面可以提供给用户采纳的候选元数据——不存储任何内容，`settings.yaml` 仍然是决定路由服务内容的唯一事实。
|
||||
|
||||
### 失败与恢复
|
||||
|
||||
@@ -212,6 +212,7 @@ pi-ai 事件变成 harness 的推理、文本、工具调用、用量与 finish
|
||||
- **分层合并对字典键没有删除**——base 声明的 `reasoningEfforts` 等级、`modelOverrides` 条目或 `compat` 字段可以被用户层覆盖，但不能被移除。
|
||||
- **`headers` 可以携带 redactor 永远看不到的凭据**——profile 解析会拒绝 Fetch 无法表示的名称与值，但该字典仍是纯字符串；以 `apiKeyEnv` 引用存储凭据。
|
||||
- **路由目录不会自行刷新**——目录就是 `settings.yaml` 的内容；这里没有任何机制向提供方查询它提供的模型。
|
||||
- **Anthropic 模型发现最多读取 1,000 个模型**——请求使用 API 的最大页大小，但不会遍历 `has_more`；第一页之外的条目需要手工添加。
|
||||
- **每条路由一种协议格式**——混合协议目录路由无法承载另一协议格式的模型；把提供方拆到两个路由键是变通办法。
|
||||
- **模态声明不受校验**——声明 `image` 而其网关不支持的模型会在提示词准入后被提供方拒绝。持久图片仍留在历史中，同一误声明模型可能再次失败；切换到纯文本模型仍然可行，因为共享 LLM 运行时会针对该请求把图片引用投影为稳定文本。
|
||||
- **未认证路由取决于其协议**——不点名凭据的路由解析为已配置但无密钥，但 pi-ai 的 OpenAI 兼容实现仍要求 API 密钥或 `Authorization` 标头，因此无密钥本地服务器需要由 `apiKeyEnv` 引用或 `headers` 中的 `Authorization` 条目提供的占位凭据。
|
||||
|
||||
@@ -13,11 +13,11 @@
|
||||
* metadata the surface offers for adoption. `settings.yaml` remains the only
|
||||
* thing that decides what a route serves.
|
||||
*
|
||||
* Only OpenAI-compatible protocols are interrogated. Their listing is the one
|
||||
* shape a gateway, a self-hosted server, and the official endpoints all agree
|
||||
* on, which is the case this action exists for; every other protocol reports
|
||||
* that it cannot be interrogated so the surface falls back to hand-entry
|
||||
* rather than guessing a response shape.
|
||||
* OpenAI-compatible and Anthropic Messages protocols are interrogated through
|
||||
* their native model-listing endpoints. The parser accepts the standard
|
||||
* `data` array and the enriched `models` map some compatible gateways expose.
|
||||
* Every other protocol reports that it cannot be interrogated so the surface
|
||||
* falls back to hand-entry rather than guessing its response fields.
|
||||
*
|
||||
* @module dsh-llm-pi-ai/discovery
|
||||
*/
|
||||
@@ -28,18 +28,26 @@ import { attributionHeaders } from '@deepseek-ai/dsh-llm'
|
||||
import { catalogModels } from './catalog.ts'
|
||||
|
||||
/**
|
||||
* Protocols whose model listing this module can read: the two that speak
|
||||
* OpenAI's `GET /models` shape with bearer auth. Azure is absent despite its
|
||||
* OpenAI lineage — it authenticates with an `api-key` header and requires an
|
||||
* `api-version` query — and Codex authenticates through OAuth; guessing at
|
||||
* either would report an authentication failure as a provider with no models.
|
||||
* pi-ai's remaining protocols are absent for the same reason.
|
||||
* Protocols whose model listing this module can read. OpenAI protocols use
|
||||
* bearer auth at `GET {baseURL}/models`; Anthropic Messages uses `x-api-key`
|
||||
* and `anthropic-version` at its native `GET /v1/models`. Azure is absent
|
||||
* despite its OpenAI lineage — it authenticates with an `api-key` header and
|
||||
* requires an `api-version` query — and Codex authenticates through OAuth;
|
||||
* guessing at either would report an authentication failure as a provider
|
||||
* with no models. pi-ai's remaining protocols are absent for the same reason.
|
||||
*/
|
||||
const LISTABLE_PROTOCOLS: ReadonlySet<string> = new Set([
|
||||
'anthropic-messages',
|
||||
'openai-completions',
|
||||
'openai-responses',
|
||||
])
|
||||
|
||||
/** Stable API version required by Anthropic's model-listing endpoint. */
|
||||
const ANTHROPIC_VERSION = '2023-06-01'
|
||||
|
||||
/** Largest model-list page accepted by Anthropic's public endpoint; discovery reads one page and does not follow `has_more`. */
|
||||
const ANTHROPIC_MODEL_LIMIT = 1000
|
||||
|
||||
/**
|
||||
* Endpoint replies larger than this are refused. The endpoint is whatever URL
|
||||
* the user typed, so the ceiling holds on the bytes actually read rather than
|
||||
@@ -49,16 +57,34 @@ const LISTABLE_PROTOCOLS: ReadonlySet<string> = new Set([
|
||||
*/
|
||||
const MAX_RESPONSE_BYTES = 4 * 1024 * 1024
|
||||
|
||||
/** One entry of an OpenAI-compatible `GET /models` reply. */
|
||||
/** Capacity fields nested by enriched model-directory replies. */
|
||||
interface ListingLimit {
|
||||
context?: unknown
|
||||
output?: unknown
|
||||
}
|
||||
|
||||
/** Per-route capacities OpenRouter nests under each entry. */
|
||||
interface ListingTopProvider {
|
||||
max_completion_tokens?: unknown
|
||||
}
|
||||
|
||||
/** One entry of a supported `GET /models` reply. */
|
||||
interface ListingEntry {
|
||||
id?: unknown
|
||||
/** Common gateway extensions; absent from the official listings. */
|
||||
name?: unknown
|
||||
display_name?: unknown
|
||||
displayName?: unknown
|
||||
contextWindow?: unknown
|
||||
context_window?: unknown
|
||||
context_length?: unknown
|
||||
max_input_tokens?: unknown
|
||||
maxOutputTokens?: unknown
|
||||
max_tokens?: unknown
|
||||
max_output_tokens?: unknown
|
||||
maxTokens?: unknown
|
||||
limit?: ListingLimit | null
|
||||
top_provider?: ListingTopProvider | null
|
||||
}
|
||||
|
||||
/** A positive integer field of a listing entry, or `undefined` when absent or unusable. */
|
||||
@@ -78,13 +104,21 @@ function label(...candidates: readonly unknown[]): string | undefined {
|
||||
}
|
||||
|
||||
/**
|
||||
* Join the endpoint base with the listing path. The base is treated as a
|
||||
* prefix rather than a URL to resolve against, so a deployment path such as
|
||||
* `https://gateway.example/openai/v1` keeps its segments instead of losing
|
||||
* them to `URL` resolution.
|
||||
* Join the endpoint base with the protocol's listing path. The base is
|
||||
* treated as a prefix rather than a URL to resolve against, so a deployment
|
||||
* path such as `https://gateway.example/openai/v1` keeps its segments instead
|
||||
* of losing them to `URL` resolution. OpenAI protocols list at
|
||||
* `{baseURL}/models`. Anthropic lists at `{root}/v1/models`, where the root is
|
||||
* the base without trailing slashes and without one trailing `/v1` segment:
|
||||
* gateway documentation publishes both spellings of the same root. Only this
|
||||
* listing URL normalizes that segment; model requests receive the configured
|
||||
* `baseURL` unchanged.
|
||||
*/
|
||||
function listingUrl(baseURL: string): string {
|
||||
return `${baseURL.replace(/\/+$/, '')}/models`
|
||||
function listingUrl(baseURL: string, api: string): string {
|
||||
const base = baseURL.replace(/\/+$/, '')
|
||||
if (api !== 'anthropic-messages') return `${base}/models`
|
||||
const root = base.endsWith('/v1') ? base.slice(0, -3) : base
|
||||
return `${root}/v1/models?limit=${String(ANTHROPIC_MODEL_LIMIT)}`
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -131,29 +165,63 @@ async function readBounded(response: Response, url: string): Promise<string> {
|
||||
}
|
||||
|
||||
/**
|
||||
* Read one OpenAI-compatible listing reply. Entries without a usable id are
|
||||
* skipped rather than failing the whole interrogation: a single malformed row
|
||||
* should not deny the user the rest of a working endpoint's catalog.
|
||||
* Read one supported model-listing reply. The standard `data` array takes
|
||||
* precedence when both supported formats are present. An enriched `models`
|
||||
* map uses each property key as the endpoint-facing id; its nested `id` is
|
||||
* only a fallback for an empty key because gateways may put a canonical model
|
||||
* identity there instead of the alias they accept on requests. Only
|
||||
* object-valued map entries are models; primitive properties are ignored
|
||||
* because they may be directory metadata rather than model records.
|
||||
*
|
||||
* Entries without a usable id are skipped rather than failing the whole
|
||||
* interrogation: a single malformed row should not deny the user the rest of
|
||||
* a working endpoint's catalog. Missing names fall back to the adopted id so
|
||||
* the Web form receives a complete human-readable row.
|
||||
*/
|
||||
function readListing(body: unknown): LlmDiscoveredModel[] {
|
||||
const data = (body as { data?: unknown } | null)?.data
|
||||
if (!Array.isArray(data)) {
|
||||
throw new LlmError(
|
||||
'the endpoint\'s model listing has no "data" array; enter this provider\'s models by hand',
|
||||
'DISCOVERY_FAILED',
|
||||
)
|
||||
const listing = body as { data?: unknown; models?: unknown } | null
|
||||
const data = listing?.data
|
||||
let listed: { readonly key?: string; readonly raw: unknown }[]
|
||||
if (Array.isArray(data)) {
|
||||
const rows = data as readonly unknown[]
|
||||
listed = rows.map(raw => ({ raw }))
|
||||
} else {
|
||||
const models = listing?.models
|
||||
if (models === null || typeof models !== 'object' || Array.isArray(models)) {
|
||||
throw new LlmError(
|
||||
'the endpoint\'s model listing has neither a "data" array nor a "models" object; '
|
||||
+ 'enter this provider\'s models by hand',
|
||||
'DISCOVERY_FAILED',
|
||||
)
|
||||
}
|
||||
listed = Object.entries(models as Record<string, unknown>)
|
||||
.filter(([, raw]) => raw !== null && typeof raw === 'object' && !Array.isArray(raw))
|
||||
.map(([key, raw]) => ({ key, raw }))
|
||||
}
|
||||
const models: LlmDiscoveredModel[] = []
|
||||
for (const raw of data) {
|
||||
for (const { key, raw } of listed) {
|
||||
const entry = raw as ListingEntry | null
|
||||
const id = label(entry?.id)
|
||||
const id = label(key, entry?.id)
|
||||
if (id === undefined) continue
|
||||
const name = label(entry?.name, entry?.display_name)
|
||||
const contextWindow = capacity(entry?.context_window, entry?.context_length)
|
||||
const maxTokens = capacity(entry?.max_output_tokens, entry?.max_tokens)
|
||||
const name = label(entry?.name, entry?.display_name, entry?.displayName) ?? id
|
||||
const contextWindow = capacity(
|
||||
entry?.contextWindow,
|
||||
entry?.context_window,
|
||||
entry?.context_length,
|
||||
entry?.max_input_tokens,
|
||||
entry?.limit?.context,
|
||||
)
|
||||
const maxTokens = capacity(
|
||||
entry?.maxOutputTokens,
|
||||
entry?.max_output_tokens,
|
||||
entry?.maxTokens,
|
||||
entry?.max_tokens,
|
||||
entry?.limit?.output,
|
||||
entry?.top_provider?.max_completion_tokens,
|
||||
)
|
||||
models.push({
|
||||
id,
|
||||
...name === undefined ? {} : { name },
|
||||
name,
|
||||
...contextWindow === undefined ? {} : { contextWindow },
|
||||
...maxTokens === undefined ? {} : { maxTokens },
|
||||
})
|
||||
@@ -235,7 +303,7 @@ export async function discoverModels(
|
||||
'DISCOVERY_UNSUPPORTED',
|
||||
)
|
||||
}
|
||||
const url = listingUrl(request.baseURL)
|
||||
const url = listingUrl(request.baseURL, api)
|
||||
// A key typed into the form wins: it may replace the stored key that is
|
||||
// failing. The stored profile is asked past the catalog and protocol checks,
|
||||
// and its credential resolver remains lazy so a typed key cannot fail over a
|
||||
@@ -248,7 +316,12 @@ export async function discoverModels(
|
||||
try {
|
||||
const headers = new Headers(stored?.headers === undefined ? undefined : Object.entries(stored.headers))
|
||||
headers.set('accept', 'application/json')
|
||||
if (apiKey !== undefined) headers.set('authorization', `Bearer ${apiKey}`)
|
||||
if (api === 'anthropic-messages') {
|
||||
headers.set('anthropic-version', ANTHROPIC_VERSION)
|
||||
if (apiKey !== undefined) headers.set('x-api-key', apiKey)
|
||||
} else if (apiKey !== undefined) {
|
||||
headers.set('authorization', `Bearer ${apiKey}`)
|
||||
}
|
||||
for (const [name, value] of Object.entries(attributionHeaders())) headers.set(name, value)
|
||||
response = await fetch(url, {
|
||||
method: 'GET',
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { readFile } from 'node:fs/promises'
|
||||
import { createServer } from 'node:http'
|
||||
import type { IncomingMessage, Server, ServerResponse } from 'node:http'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
@@ -111,6 +112,9 @@ describe('draft-provider model discovery', () => {
|
||||
body: JSON.stringify({
|
||||
data: [
|
||||
{ id: 'acme-large', display_name: 'Acme Large', context_length: 65_536, max_output_tokens: 4096 },
|
||||
{ id: 'acme-camel', displayName: 'Acme Camel', contextWindow: 131_072, maxOutputTokens: 8192 },
|
||||
{ id: 'acme-mixed', name: 'Acme Mixed', context_window: 32_768, maxTokens: 2048 },
|
||||
{ id: 'acme-legacy', max_tokens: 1024 },
|
||||
{ id: 'acme-small' },
|
||||
],
|
||||
}),
|
||||
@@ -121,13 +125,106 @@ describe('draft-provider model discovery', () => {
|
||||
|
||||
expect(models).toEqual([
|
||||
{ id: 'acme-large', name: 'Acme Large', contextWindow: 65_536, maxTokens: 4096 },
|
||||
{ id: 'acme-small' },
|
||||
{ id: 'acme-camel', name: 'Acme Camel', contextWindow: 131_072, maxTokens: 8192 },
|
||||
{ id: 'acme-mixed', name: 'Acme Mixed', contextWindow: 32_768, maxTokens: 2048 },
|
||||
{ id: 'acme-legacy', name: 'acme-legacy', maxTokens: 1024 },
|
||||
{ id: 'acme-small', name: 'acme-small' },
|
||||
])
|
||||
expect(server.paths).toEqual(['/v1/models'])
|
||||
expect(server.headers[0]?.authorization).toBe('Bearer probe-key')
|
||||
expect(server.headers[0]?.['user-agent']).toBe(userAgent())
|
||||
})
|
||||
|
||||
it('reads an enriched models map using route ids and nested capacities', async () => {
|
||||
const server = await listingServer({
|
||||
body: JSON.stringify({
|
||||
models: {
|
||||
'lobechat-deepseek-chat': {
|
||||
id: 'deepseek/deepseek-v4-flash',
|
||||
name: 'DeepSeek V4 Flash',
|
||||
limit: { context: 1_048_576, output: 384_000 },
|
||||
},
|
||||
'bare-route': {},
|
||||
'': { id: 'nested-id', display_name: 'Nested fallback' },
|
||||
'malformed-route': null,
|
||||
'primitive-route': 'not a model record',
|
||||
},
|
||||
}),
|
||||
})
|
||||
const ctx = await harness()
|
||||
|
||||
expect(await ctx.llm.discoverModels('llm-pi-ai', { baseURL: server.url })).toEqual([
|
||||
{
|
||||
id: 'lobechat-deepseek-chat',
|
||||
name: 'DeepSeek V4 Flash',
|
||||
contextWindow: 1_048_576,
|
||||
maxTokens: 384_000,
|
||||
},
|
||||
{ id: 'bare-route', name: 'bare-route' },
|
||||
{ id: 'nested-id', name: 'Nested fallback' },
|
||||
])
|
||||
})
|
||||
|
||||
it('uses Anthropic model-listing paths, headers, and capacity fields', async () => {
|
||||
const server = await listingServer({
|
||||
body: JSON.stringify({
|
||||
data: [
|
||||
{
|
||||
id: 'claude-sonnet',
|
||||
display_name: 'Claude Sonnet',
|
||||
max_input_tokens: 200_000,
|
||||
max_tokens: 64_000,
|
||||
},
|
||||
],
|
||||
}),
|
||||
})
|
||||
const ctx = await harness()
|
||||
|
||||
const rootModels = await ctx.llm.discoverModels('llm-pi-ai', {
|
||||
baseURL: server.url,
|
||||
api: 'anthropic-messages',
|
||||
apiKey: 'anthropic-key',
|
||||
})
|
||||
const versionedModels = await ctx.llm.discoverModels('llm-pi-ai', {
|
||||
baseURL: `${server.url}/v1`,
|
||||
api: 'anthropic-messages',
|
||||
apiKey: 'anthropic-key',
|
||||
})
|
||||
await ctx.llm.discoverModels('llm-pi-ai', {
|
||||
baseURL: server.url,
|
||||
api: 'anthropic-messages',
|
||||
})
|
||||
|
||||
expect(rootModels).toEqual([
|
||||
{ id: 'claude-sonnet', name: 'Claude Sonnet', contextWindow: 200_000, maxTokens: 64_000 },
|
||||
])
|
||||
expect(versionedModels).toEqual(rootModels)
|
||||
expect(server.paths).toEqual([
|
||||
'/v1/models?limit=1000',
|
||||
'/v1/models?limit=1000',
|
||||
'/v1/models?limit=1000',
|
||||
])
|
||||
expect(server.headers.map(headers => headers['x-api-key']))
|
||||
.toEqual(['anthropic-key', 'anthropic-key', undefined])
|
||||
expect(server.headers.map(headers => headers['anthropic-version']))
|
||||
.toEqual(['2023-06-01', '2023-06-01', '2023-06-01'])
|
||||
expect(server.headers.map(headers => headers.authorization)).toEqual([undefined, undefined, undefined])
|
||||
expect(server.headers.map(headers => headers['user-agent'])).toEqual([userAgent(), userAgent(), userAgent()])
|
||||
})
|
||||
|
||||
it('prefers the standard data array when both supported formats are present', async () => {
|
||||
const server = await listingServer({
|
||||
body: JSON.stringify({
|
||||
data: [{ id: 'standard' }],
|
||||
models: { enriched: { name: 'Enriched' } },
|
||||
}),
|
||||
})
|
||||
const ctx = await harness()
|
||||
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { baseURL: server.url }))
|
||||
.resolves.toEqual([{ id: 'standard', name: 'standard' }])
|
||||
})
|
||||
|
||||
it('keeps a deployment path instead of resolving it away', async () => {
|
||||
const server = await listingServer({ body: JSON.stringify({ data: [{ id: 'm' }] }) })
|
||||
const ctx = await harness()
|
||||
@@ -220,7 +317,7 @@ describe('draft-provider model discovery', () => {
|
||||
const ctx = await harness()
|
||||
|
||||
expect(await ctx.llm.discoverModels('llm-pi-ai', { baseURL: server.url }))
|
||||
.toEqual([{ id: 'good' }, { id: 'zero-capacity' }])
|
||||
.toEqual([{ id: 'good', name: 'good' }, { id: 'zero-capacity', name: 'zero-capacity' }])
|
||||
})
|
||||
|
||||
it('points at the credential for a rejected one, and only then', async () => {
|
||||
@@ -244,7 +341,7 @@ describe('draft-provider model discovery', () => {
|
||||
const ctx = await harness()
|
||||
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { baseURL: server.url }))
|
||||
.rejects.toThrow(/no "data" array; enter this provider's models by hand/)
|
||||
.rejects.toThrow(/neither a "data" array nor a "models" object/)
|
||||
|
||||
const broken = await listingServer({ body: 'not json at all' })
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { baseURL: broken.url }))
|
||||
@@ -274,7 +371,7 @@ describe('draft-provider model discovery', () => {
|
||||
.rejects.toMatchObject({ code: 'DISCOVERY_FAILED' })
|
||||
})
|
||||
|
||||
it.each(['anthropic-messages', 'azure-openai-responses', 'openai-codex-responses', 'google-generative-ai'])(
|
||||
it.each(['azure-openai-responses', 'openai-codex-responses', 'google-generative-ai'])(
|
||||
'says it cannot interrogate %s rather than guessing a shape',
|
||||
async (api) => {
|
||||
// Azure authenticates with an `api-key` header and an `api-version`
|
||||
@@ -386,3 +483,61 @@ describe('probe key format', () => {
|
||||
expect(headers.has('authorization')).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
/**
|
||||
* Replies recorded from live endpoints on 2026-09-02, plus the reply
|
||||
* Anthropic's List Models reference documents. Each file keeps the reply's
|
||||
* top-level fields and entry objects verbatim; only a recorded entry list is
|
||||
* cut down to the named entries so the archive stays small.
|
||||
*/
|
||||
const RECORDED_LISTINGS = [
|
||||
{
|
||||
name: 'OpenRouter GET /api/v1/models',
|
||||
file: 'openrouter-2026-09-02.json',
|
||||
api: 'openai-completions',
|
||||
models: [
|
||||
{ id: 'anthropic/claude-fable-5.1', name: 'Anthropic: Claude Fable 5.1', contextWindow: 1_000_000, maxTokens: 128_000 },
|
||||
// The router's own aggregate route reports no completion cap.
|
||||
{ id: 'openrouter/auto-beta', name: 'Auto Router (Beta)', contextWindow: 2_000_000 },
|
||||
{ id: 'deepseek/deepseek-v4-flash', name: 'DeepSeek: DeepSeek V4 Flash 0423', contextWindow: 1_048_576, maxTokens: 384_000 },
|
||||
],
|
||||
},
|
||||
{
|
||||
name: 'the models.dev anthropic provider object',
|
||||
file: 'models-dev-anthropic-2026-09-02.json',
|
||||
api: 'openai-completions',
|
||||
models: [
|
||||
{ id: 'claude-opus-4-7', name: 'Claude Opus 4.7', contextWindow: 1_000_000, maxTokens: 128_000 },
|
||||
{ id: 'claude-fable-5-1', name: 'Claude Fable 5.1', contextWindow: 1_000_000, maxTokens: 128_000 },
|
||||
{ id: 'claude-haiku-4-5', name: 'Claude Haiku 4.5 (latest)', contextWindow: 200_000, maxTokens: 64_000 },
|
||||
],
|
||||
},
|
||||
{
|
||||
name: 'DeepSeek GET /models',
|
||||
file: 'deepseek-2026-09-02.json',
|
||||
api: 'openai-completions',
|
||||
models: [
|
||||
{ id: 'deepseek-v4-flash', name: 'deepseek-v4-flash' },
|
||||
{ id: 'deepseek-v4-pro', name: 'deepseek-v4-pro' },
|
||||
{ id: 'deepseek-v4-flash-vision-exp', name: 'deepseek-v4-flash-vision-exp' },
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "Anthropic's documented GET /v1/models example",
|
||||
file: 'anthropic-reference-example.json',
|
||||
api: 'anthropic-messages',
|
||||
// The reference example fills both capacities with 0, which is not a
|
||||
// usable capacity, so the row carries the name alone.
|
||||
models: [{ id: 'claude-opus-5', name: 'Claude Opus 5' }],
|
||||
},
|
||||
]
|
||||
|
||||
describe('recorded provider listings', () => {
|
||||
it.each(RECORDED_LISTINGS)('reads $name as recorded', async ({ file, api, models }) => {
|
||||
const body = await readFile(new URL(`./fixtures/model-listings/${file}`, import.meta.url), 'utf8')
|
||||
const server = await listingServer({ body })
|
||||
const ctx = await harness()
|
||||
|
||||
await expect(ctx.llm.discoverModels('llm-pi-ai', { baseURL: server.url, api })).resolves.toEqual(models)
|
||||
})
|
||||
})
|
||||
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
{
|
||||
"data": [
|
||||
{
|
||||
"id": "claude-opus-5",
|
||||
"capabilities": {
|
||||
"batch": {
|
||||
"supported": true
|
||||
},
|
||||
"citations": {
|
||||
"supported": true
|
||||
},
|
||||
"code_execution": {
|
||||
"supported": true
|
||||
},
|
||||
"context_management": {
|
||||
"clear_thinking_20251015": {
|
||||
"supported": true
|
||||
},
|
||||
"clear_tool_uses_20250919": {
|
||||
"supported": true
|
||||
},
|
||||
"compact_20260112": {
|
||||
"supported": true
|
||||
},
|
||||
"supported": true
|
||||
},
|
||||
"effort": {
|
||||
"high": {
|
||||
"supported": true
|
||||
},
|
||||
"low": {
|
||||
"supported": true
|
||||
},
|
||||
"max": {
|
||||
"supported": true
|
||||
},
|
||||
"medium": {
|
||||
"supported": true
|
||||
},
|
||||
"supported": true,
|
||||
"xhigh": {
|
||||
"supported": true
|
||||
}
|
||||
},
|
||||
"image_input": {
|
||||
"supported": true
|
||||
},
|
||||
"pdf_input": {
|
||||
"supported": true
|
||||
},
|
||||
"structured_outputs": {
|
||||
"supported": true
|
||||
},
|
||||
"thinking": {
|
||||
"supported": true,
|
||||
"types": {
|
||||
"adaptive": {
|
||||
"supported": true
|
||||
},
|
||||
"enabled": {
|
||||
"supported": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"created_at": "2026-07-24T00:00:00Z",
|
||||
"display_name": "Claude Opus 5",
|
||||
"max_input_tokens": 0,
|
||||
"max_tokens": 0,
|
||||
"type": "model"
|
||||
}
|
||||
],
|
||||
"first_id": "first_id",
|
||||
"has_more": true,
|
||||
"last_id": "last_id"
|
||||
}
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "deepseek-v4-flash",
|
||||
"object": "model",
|
||||
"owned_by": "deepseek"
|
||||
},
|
||||
{
|
||||
"id": "deepseek-v4-pro",
|
||||
"object": "model",
|
||||
"owned_by": "deepseek"
|
||||
},
|
||||
{
|
||||
"id": "deepseek-v4-flash-vision-exp",
|
||||
"object": "model",
|
||||
"owned_by": "deepseek"
|
||||
}
|
||||
]
|
||||
}
|
||||
+146
@@ -0,0 +1,146 @@
|
||||
{
|
||||
"id": "anthropic",
|
||||
"env": [
|
||||
"ANTHROPIC_API_KEY"
|
||||
],
|
||||
"npm": "@ai-sdk/anthropic",
|
||||
"name": "Anthropic",
|
||||
"doc": "https://docs.anthropic.com/en/docs/about-claude/models",
|
||||
"models": {
|
||||
"claude-opus-4-7": {
|
||||
"id": "claude-opus-4-7",
|
||||
"name": "Claude Opus 4.7",
|
||||
"description": "Stronger Opus tier for advanced software work and high-stakes reasoning",
|
||||
"family": "claude-opus",
|
||||
"attachment": true,
|
||||
"reasoning": true,
|
||||
"reasoning_options": [
|
||||
{
|
||||
"type": "effort",
|
||||
"values": [
|
||||
"low",
|
||||
"medium",
|
||||
"high",
|
||||
"xhigh",
|
||||
"max"
|
||||
]
|
||||
}
|
||||
],
|
||||
"tool_call": true,
|
||||
"structured_output": true,
|
||||
"temperature": false,
|
||||
"knowledge": "2026-01-31",
|
||||
"release_date": "2026-04-14",
|
||||
"last_updated": "2026-04-16",
|
||||
"modalities": {
|
||||
"input": [
|
||||
"text",
|
||||
"image",
|
||||
"pdf"
|
||||
],
|
||||
"output": [
|
||||
"text"
|
||||
]
|
||||
},
|
||||
"open_weights": false,
|
||||
"limit": {
|
||||
"context": 1000000,
|
||||
"output": 128000
|
||||
},
|
||||
"cost": {
|
||||
"input": 5,
|
||||
"output": 25,
|
||||
"cache_read": 0.5,
|
||||
"cache_write": 6.25
|
||||
}
|
||||
},
|
||||
"claude-fable-5-1": {
|
||||
"id": "claude-fable-5-1",
|
||||
"name": "Claude Fable 5.1",
|
||||
"description": "Claude model for demanding reasoning and long-horizon agentic work",
|
||||
"family": "claude-fable",
|
||||
"attachment": true,
|
||||
"reasoning": true,
|
||||
"reasoning_options": [
|
||||
{
|
||||
"type": "effort",
|
||||
"values": [
|
||||
"low",
|
||||
"medium",
|
||||
"high",
|
||||
"xhigh",
|
||||
"max"
|
||||
]
|
||||
}
|
||||
],
|
||||
"tool_call": true,
|
||||
"structured_output": true,
|
||||
"temperature": false,
|
||||
"knowledge": "2026-06",
|
||||
"release_date": "2026-09-01",
|
||||
"last_updated": "2026-09-01",
|
||||
"modalities": {
|
||||
"input": [
|
||||
"text",
|
||||
"image",
|
||||
"pdf"
|
||||
],
|
||||
"output": [
|
||||
"text"
|
||||
]
|
||||
},
|
||||
"open_weights": false,
|
||||
"limit": {
|
||||
"context": 1000000,
|
||||
"output": 128000
|
||||
},
|
||||
"cost": {
|
||||
"input": 10,
|
||||
"output": 50,
|
||||
"cache_read": 0.25,
|
||||
"cache_write": 12.5
|
||||
}
|
||||
},
|
||||
"claude-haiku-4-5": {
|
||||
"id": "claude-haiku-4-5",
|
||||
"name": "Claude Haiku 4.5 (latest)",
|
||||
"description": "Fast Claude lane for lightweight agents, office tasks, and responsive chat",
|
||||
"family": "claude-haiku",
|
||||
"attachment": true,
|
||||
"reasoning": true,
|
||||
"reasoning_options": [
|
||||
{
|
||||
"type": "budget_tokens",
|
||||
"min": 1024
|
||||
}
|
||||
],
|
||||
"tool_call": true,
|
||||
"structured_output": true,
|
||||
"temperature": true,
|
||||
"knowledge": "2025-02-28",
|
||||
"release_date": "2025-10-15",
|
||||
"last_updated": "2025-10-15",
|
||||
"modalities": {
|
||||
"input": [
|
||||
"text",
|
||||
"image",
|
||||
"pdf"
|
||||
],
|
||||
"output": [
|
||||
"text"
|
||||
]
|
||||
},
|
||||
"open_weights": false,
|
||||
"limit": {
|
||||
"context": 200000,
|
||||
"output": 64000
|
||||
},
|
||||
"cost": {
|
||||
"input": 1,
|
||||
"output": 5,
|
||||
"cache_read": 0.1,
|
||||
"cache_write": 1.25
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+299
@@ -0,0 +1,299 @@
|
||||
{
|
||||
"data": [
|
||||
{
|
||||
"id": "anthropic/claude-fable-5.1",
|
||||
"canonical_slug": "anthropic/claude-fable-5.1-20260831",
|
||||
"hugging_face_id": null,
|
||||
"name": "Anthropic: Claude Fable 5.1",
|
||||
"created": 1788285838,
|
||||
"description": "Claude Fable 5.1 improves on Claude Fable 5 across the board, with the biggest gains in agentic coding, long-running agentic workflows, and knowledge work: long code refactors, front-end and visual...",
|
||||
"context_length": 1000000,
|
||||
"architecture": {
|
||||
"modality": "text+image+file->text",
|
||||
"input_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"file"
|
||||
],
|
||||
"output_modalities": [
|
||||
"text"
|
||||
],
|
||||
"tokenizer": "Claude",
|
||||
"instruct_type": null
|
||||
},
|
||||
"pricing": {
|
||||
"prompt": "0.00001",
|
||||
"completion": "0.00005",
|
||||
"web_search": "0.01",
|
||||
"input_cache_read": "0.00000025",
|
||||
"input_cache_write": "0.0000125",
|
||||
"input_cache_write_1h": "0.00002"
|
||||
},
|
||||
"top_provider": {
|
||||
"context_length": 1000000,
|
||||
"max_completion_tokens": 128000,
|
||||
"is_moderated": true
|
||||
},
|
||||
"per_request_limits": null,
|
||||
"supported_parameters": [
|
||||
"include_reasoning",
|
||||
"max_completion_tokens",
|
||||
"max_tokens",
|
||||
"reasoning",
|
||||
"reasoning_effort",
|
||||
"response_format",
|
||||
"stop",
|
||||
"structured_outputs",
|
||||
"tools",
|
||||
"verbosity"
|
||||
],
|
||||
"default_parameters": {},
|
||||
"supported_voices": null,
|
||||
"knowledge_cutoff": null,
|
||||
"expiration_date": null,
|
||||
"links": {
|
||||
"details": "/api/v1/models/anthropic/claude-fable-5.1-20260831/endpoints"
|
||||
},
|
||||
"benchmarks": {
|
||||
"design_arena": [
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "website",
|
||||
"elo": 1351,
|
||||
"win_rate": 58.5,
|
||||
"rank": 2
|
||||
}
|
||||
],
|
||||
"artificial_analysis": {
|
||||
"intelligence_index": 65.7,
|
||||
"coding_index": 81.6,
|
||||
"agentic_index": 61.3
|
||||
}
|
||||
},
|
||||
"reasoning": {
|
||||
"mandatory": true,
|
||||
"supported_efforts": [
|
||||
"max",
|
||||
"xhigh",
|
||||
"high",
|
||||
"medium",
|
||||
"low"
|
||||
],
|
||||
"default_effort": "high"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "openrouter/auto-beta",
|
||||
"canonical_slug": "openrouter/auto-beta",
|
||||
"hugging_face_id": null,
|
||||
"name": "Auto Router (Beta)",
|
||||
"created": 1784311165,
|
||||
"description": "Auto Router (Beta) is a task-aware router from OpenRouter. It classifies each request, then routes it the [most popular model](/rankings#task-spend) for that task based on aggregate spend, filtered by your...",
|
||||
"context_length": 2000000,
|
||||
"architecture": {
|
||||
"modality": "text+image+file+audio+video->text+image",
|
||||
"input_modalities": [
|
||||
"text",
|
||||
"image",
|
||||
"audio",
|
||||
"file",
|
||||
"video"
|
||||
],
|
||||
"output_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"tokenizer": "Router",
|
||||
"instruct_type": null
|
||||
},
|
||||
"pricing": {
|
||||
"prompt": "-1",
|
||||
"completion": "-1"
|
||||
},
|
||||
"top_provider": {
|
||||
"context_length": null,
|
||||
"max_completion_tokens": null,
|
||||
"is_moderated": false
|
||||
},
|
||||
"per_request_limits": null,
|
||||
"supported_parameters": [
|
||||
"frequency_penalty",
|
||||
"include_reasoning",
|
||||
"logit_bias",
|
||||
"logprobs",
|
||||
"max_tokens",
|
||||
"min_p",
|
||||
"prediction",
|
||||
"presence_penalty",
|
||||
"reasoning",
|
||||
"reasoning_effort",
|
||||
"repetition_penalty",
|
||||
"response_format",
|
||||
"seed",
|
||||
"stop",
|
||||
"structured_outputs",
|
||||
"temperature",
|
||||
"tool_choice",
|
||||
"tools",
|
||||
"top_a",
|
||||
"top_k",
|
||||
"top_logprobs",
|
||||
"top_p",
|
||||
"web_search_options"
|
||||
],
|
||||
"default_parameters": {
|
||||
"temperature": null,
|
||||
"top_p": null,
|
||||
"top_k": null,
|
||||
"frequency_penalty": null,
|
||||
"presence_penalty": null,
|
||||
"repetition_penalty": null
|
||||
},
|
||||
"supported_voices": null,
|
||||
"knowledge_cutoff": null,
|
||||
"expiration_date": null,
|
||||
"links": {
|
||||
"details": "/api/v1/models/openrouter/auto-beta/endpoints"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "deepseek/deepseek-v4-flash",
|
||||
"canonical_slug": "deepseek/deepseek-v4-flash-20260423",
|
||||
"hugging_face_id": "deepseek-ai/DeepSeek-V4-Flash",
|
||||
"name": "DeepSeek: DeepSeek V4 Flash 0423",
|
||||
"created": 1777000666,
|
||||
"description": "DeepSeek V4 Flash is an efficiency-optimized Mixture-of-Experts model from DeepSeek with 284B total parameters and 13B activated parameters, supporting a 1M-token context window. It is designed for fast inference and...",
|
||||
"context_length": 1048576,
|
||||
"architecture": {
|
||||
"modality": "text->text",
|
||||
"input_modalities": [
|
||||
"text"
|
||||
],
|
||||
"output_modalities": [
|
||||
"text"
|
||||
],
|
||||
"tokenizer": "DeepSeek",
|
||||
"instruct_type": null
|
||||
},
|
||||
"pricing": {
|
||||
"prompt": "0.00000008708",
|
||||
"completion": "0.00000017416",
|
||||
"input_cache_read": "0.000000017416"
|
||||
},
|
||||
"top_provider": {
|
||||
"context_length": 1024000,
|
||||
"max_completion_tokens": 384000,
|
||||
"is_moderated": false
|
||||
},
|
||||
"per_request_limits": null,
|
||||
"supported_parameters": [
|
||||
"frequency_penalty",
|
||||
"include_reasoning",
|
||||
"logit_bias",
|
||||
"logprobs",
|
||||
"max_completion_tokens",
|
||||
"max_tokens",
|
||||
"min_p",
|
||||
"presence_penalty",
|
||||
"reasoning",
|
||||
"reasoning_effort",
|
||||
"repetition_penalty",
|
||||
"response_format",
|
||||
"seed",
|
||||
"stop",
|
||||
"structured_outputs",
|
||||
"temperature",
|
||||
"tool_choice",
|
||||
"tools",
|
||||
"top_a",
|
||||
"top_k",
|
||||
"top_logprobs",
|
||||
"top_p"
|
||||
],
|
||||
"default_parameters": {},
|
||||
"supported_voices": null,
|
||||
"knowledge_cutoff": null,
|
||||
"expiration_date": null,
|
||||
"links": {
|
||||
"details": "/api/v1/models/deepseek/deepseek-v4-flash-20260423/endpoints"
|
||||
},
|
||||
"benchmarks": {
|
||||
"design_arena": [
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "3d",
|
||||
"elo": 1229,
|
||||
"win_rate": 49.3,
|
||||
"rank": 40
|
||||
},
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "asciiart",
|
||||
"elo": 1138,
|
||||
"win_rate": 42.8,
|
||||
"rank": 47
|
||||
},
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "codecategories",
|
||||
"elo": 1226,
|
||||
"win_rate": 48.9,
|
||||
"rank": 43
|
||||
},
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "dataviz",
|
||||
"elo": 1148,
|
||||
"win_rate": 40.6,
|
||||
"rank": 74
|
||||
},
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "gamedev",
|
||||
"elo": 1232,
|
||||
"win_rate": 50.2,
|
||||
"rank": 39
|
||||
},
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "svg",
|
||||
"elo": 1193,
|
||||
"win_rate": 48.4,
|
||||
"rank": 34
|
||||
},
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "uicomponent",
|
||||
"elo": 1188,
|
||||
"win_rate": 44.7,
|
||||
"rank": 58
|
||||
},
|
||||
{
|
||||
"arena": "models",
|
||||
"category": "website",
|
||||
"elo": 1224,
|
||||
"win_rate": 49.1,
|
||||
"rank": 45
|
||||
}
|
||||
],
|
||||
"artificial_analysis": {
|
||||
"intelligence_index": 42.1,
|
||||
"coding_index": 56.2,
|
||||
"agentic_index": 33.7
|
||||
}
|
||||
},
|
||||
"reasoning": {
|
||||
"mandatory": false,
|
||||
"supported_efforts": [
|
||||
"xhigh",
|
||||
"high"
|
||||
],
|
||||
"default_effort": "high"
|
||||
}
|
||||
}
|
||||
],
|
||||
"total_count": 421,
|
||||
"links": {
|
||||
"next": null
|
||||
}
|
||||
}
|
||||
@@ -151,7 +151,7 @@ describe('llm-pi-ai real dormant composition', () => {
|
||||
provider: 'acme-gateway',
|
||||
baseURL: server.url,
|
||||
api: 'openai-completions',
|
||||
})).resolves.toEqual([{ id: 'acme-private' }])
|
||||
})).resolves.toEqual([{ id: 'acme-private', name: 'acme-private' }])
|
||||
expect(server.paths).toEqual(['/models'])
|
||||
expect(server.headers[0]?.['x-company-code']).toBe('private-tenant')
|
||||
expect(server.headers[0]?.authorization).toBe('Bearer key-from-store')
|
||||
|
||||
Reference in New Issue
Block a user