mirror of https://github.com/deepseek-ai/deepseek-harness.git synced 2026-11-09 20:07:43 +08:00

Files

T

Yichen Jiang c48f8523f2 feat(llm): follow the official /models listing on the DeepSeek API-key route (#6944 )

* refactor(llm-deepseek): share the /models listing machinery

Move the account route's `/models` request and decoding, its
credential-scoped listing cache, and listed-catalog resolution with the
version gate into the shared DeepSeek Messages library, so a second
route can follow the same listing. Names drop the account prefix, the
cache takes each access's endpoint settings with the call, and it gains
`invalidate()` for a forced reload. `isBuiltInCatalog()` recognizes the
catalog an uncustomized `models` setting resolves to.

The account route keeps its behavior; only its imports and the
override diagnostic wording ("model listing") change.

* feat(llm-deepseek-api-key): follow the official /models listing

While `models` equals the built-in catalog and the Messages origin is
https://api.deepseek.com, `deepseek-official` serves the official
`GET /models` listing, authenticated with the API key in `x-api-key`.
Loading, refresh, fallback, merging, and the client-version gate match
the account route: the first access waits up to `modelsTimeoutMs`, a
listing older than `modelsMaxAgeMs` refreshes in the background on the
next access, a failure keeps the last good listing or the built-in
catalog, and listed models this build cannot serve are refused before
network I/O.

A customized `models` list replaces the catalog and requests nothing.
Switching between a customized catalog and the listing discards the
cached listing, so Restore defaults reloads it on the next access.

Suites that boot the route on the official endpoint now answer the
listing themselves instead of reaching the network.

* docs: describe the official route model listing

Document when `deepseek-official` follows the official `/models`
listing, its timing fields, customization precedence, and the forced
reload after Restore defaults in the API-key README, the shared
transport README, and the provider guide. Record the decision in a new
Agent Note, keep the account note's statement of where the shared
listing code lives current, and regenerate the configuration catalog.

* test(llm-deepseek-api-key): pin the once-per-listing hidden-entry diagnostic

* docs: regenerate module and event graphs for the shared listing

* fix(llm-deepseek-api-key): report credential failures on the model request

On the official endpoint, `connect` waited for the listing view, whose
credential lookup rethrew a malformed key or a credential-store failure
before the agent loop recorded the prompt. A failed lookup now resolves
the connection from the current listing view, so `resolveAuth` reports
the failure on the model request and the turn keeps the prompt and the
failed attempt. A caller abort still rejects.

* docs(llm-deepseek-api-key): name which hidden listing entries are logged

* test: keep unit suites off the official model listing

Suites that boot the DeepSeek routes with a dummy or inherited key now
request `GET https://api.deepseek.com/models` on the official endpoint.
A unit-lane setup file rejects that request at once, so such a suite
serves the built-in catalog instead of reaching production or waiting
out the listing deadline on a network that never answers; suites that
exercise the listing answer `fetch` themselves. Other requests pass
through unchanged.

* feat(llm-deepseek): read tool_update from the model listing

Both DeepSeek routes now take a listed model's Messages tool update mode
from `api_capabilities.anthropic_messages.tool_update`, alongside the
system prompt mode. `in-history` and `addition-only` replace the
bundled mode; any other value reads as `leading-only` and removes it.
An absent or `null` mode keeps the bundled one, so a listing that does
not declare the field leaves `deepseek-flash` on `addition-only`.

`system_prompt_update: null` likewise keeps the bundled mode now; it
previously read as `leading-only`, which a serializer writing empty
values as null would have turned into a cache-unfriendly downgrade.

* fix(llm-deepseek): sync the built-in context window with the official listing

The official `/models` listing declares a 1,048,576-token context window
for `deepseek-flash` and `deepseek-v4-pro`; the built-in catalog said
1,000,000, a decimal-versus-binary mismatch. Both built-in entries now
carry the listed value, so offline starts and listing failures size the
context the same way the listing does. The Models page shows the exact
count for these rows, since 1,048,576 has no decimal K/M spelling.

`defaultContextWindow`, the fallback for model ids the catalog does not
describe, stays 1,000,000.

* test(llm-deepseek-api-key): reject the stubbed listing request with an Error

* test(cli): keep the built SDK smoke off the official model listing

The SDK profile's initialize now resolves the deepseek-official catalog,
which requests GET https://api.deepseek.com/models on the official
endpoint. With the smoke's fixture key that request answers 401 and the
route warns on stderr, so the test's empty-stderr assertion failed in the
e2e and snapshots lanes. Point the run at a loopback mock and assert it
received no model call.

* test(llm-deepseek): compose the malformed-key turn without the native executor

release/dsh-0.2.1 predates @deepseek-ai/dsh-tool-executor-native; its tool
runtime executes tools itself. Compose the official-route turn the way
account-routing.spec.ts does on this branch.

* fix(llm-deepseek-api-key): gate nothing when the credential lookup fails

The listing cache switches scope only after it reads a credential, so a
failed lookup left `connect` serving the previous credential's listing.
After a key whose listing refuses a model was replaced by a malformed one,
`admit` refused that model with CLIENT_UPGRADE_REQUIRED before the turn
recorded the prompt, instead of `resolveAuth` reporting the credential
error on the request. Serve the built-in catalog without listing gates
when the lookup fails.

* test(llm-deepseek-api-key): pin the once-per-listing diagnostic across request preparation

The hidden-entry diagnostic test repeated the same discovery assertion to
prove a second pass reports nothing new, which read as a duplicate. Assert
the single warning after the request preparations instead; they resolve
the same listing, so dropping the per-listing deduplication still fails
the test.

* docs(llm-deepseek-api-key): note that a list matching an earlier built-in catalog counts as customized

Customization is judged against the running version's built-in catalog,
so a saved `models` list equal to an earlier version's catalog stops
following the listing once the built-in catalog changes, until Restore
defaults. Record the consequence in the Agent Note and the README's known
limitations.

2026-10-22 20:42:19 +08:00

3.9 KiB

Raw Blame History

Agent Note: DeepSeek account models come from the account listing and gate by client version

Status: implemented

English | 中文

Problem

The account route's catalog was deployment configuration: a build advertised the models it shipped with, a new model reached users only through a release, and nothing kept an older build from requesting a model it cannot drive. The account service publishes per-account model metadata at GET /models, and that listing can name models an older build does not understand.

Decision

llm-deepseek-account loads the listing lazily for each credential, waits for it once up to modelsTimeoutMs, refreshes it in the background after modelsMaxAgeMs, keeps the last good listing when a load fails or returns no model, and serves the bundled catalog before any listing loads. The cache lives in memory and is keyed by endpoint and token fingerprint, so per-account rollouts stay separate.
A listed model is available when its client_requirements["deepseek-harness"].min_version is at most the current DeepSeek Harness version in semantic-version order. Without that declaration, only ids the built-in catalog knows are available; the others stay hidden, because a client cannot tell which future version supports them. A request for a refused listed model fails before network I/O with CLIENT_UPGRADE_REQUIRED or MODEL_UNSUPPORTED.
Listed capabilities replace bundled ones field by field; the bundled entry keeps client-side knowledge the listing does not declare, such as image budgets and a protocol mode the listing omits or sets to null; modelOverrides applies last and reuses the models entry fields and validation. max_output_tokens bounds the request default instead of becoming it.
The shared transport gains only two provider hooks, connect and admit. Listing, gating, and merging live in the shared transport, which the official route also uses.

Alternatives considered

Server-side filtering by client version only. Codex sends client_version and lets its backend drop models. An older build then cannot show that a newer version supports a model, and a listing that omits a declaration would expose an unknown model.

Default parameters for unknown models. Reasonix and Pi drive unknown ids with generic defaults, and MiniMax treats that as a defect. A model that needs client changes would run with the wrong behavior.

Whole-listing validation. Codex, ZCode, and Kimi reject the whole listing on one bad field. One malformed row would then send every account back to the bundled catalog.

Persisting the last good listing. A durable cache adds a persistence format and its migration duty; the bundled catalog already covers offline starts.

Keeping models with merge-by-id semantics. The API-key route's models replaces its catalog, so the same field name with merge semantics would invert its meaning between sibling routes.

Generalizing the shared transport. Per-model effort levels, an output-capability field, and catalog-change announcements would widen the Messages package for one consumer. The listed effort levels match the bundled defaults, and a model that needs different ones needs a release, which the gate enforces.

Consequences

An older build never offers a model it does not know unless the service declares a minimum version that build satisfies, and the service must declare min_version for a future build to show that an upgrade enables a model.
A modified client can still request any id; the service enforces version floors only if it rejects such requests itself.
The first account access after sign-in may wait up to modelsTimeoutMs; an offline start uses bundled capabilities until a listing loads; a listing change during a running process reaches the Web picker at its next catalog refresh.

3.9 KiB Raw Blame History