AI Gateway Models

Varity TeamCore Contributors

The catalog is served live. This page is a snapshot of it; the endpoint is the source of truth:

curl https://ai.varity.app/v1/models

GET /v1/models is public and needs no API key. It returns an OpenAI-shaped { "object": "list", "data": [...] } response plus a top-level privacy_scope, the privacy floor the public catalog is projected at ("anonymized" on this unauthenticated route). Varity adds fields OpenAI does not return on each entry: display_name, maker, capabilities, context_window, max_completion_tokens, privacy, privacy_routes, availability, refreshed_at, and customer_pricing.

Tip

Do not hardcode the catalog. Resolve models at runtime from /v1/models and read capabilities per model. The catalog changes without a docs release.

Catalog

The catalog is not chat-only. Each entry’s capabilities.output_modalities says what the model produces: text (chat completions), embedding, image, or audio (speech). Transcription models take audio in and return text. Read availability.status per model from the endpoint rather than from this page. See Availability is withheld, not assumed for what the other statuses mean.

Pass an entry’s id verbatim as the model field of a request on the matching route. Ids are readable human slugs, for example glm-5-2, llama-3-3-70b, and gpt-4o. display_name is a separate human label, not an identifier.

maker says who built the model, never who serves it (owned_by is always varity). It carries a stable slug key and a human label; the key is unreported, with the label Maker not published, when no maker evidence was observed.

Prices are USD, taken from each model’s customer_pricing object, and the unit differs by modality: see Pricing below.

This page carries no model table on purpose. A snapshot of ids, context windows, capabilities, and prices is stale the moment the catalog changes, so query GET /v1/models and read the fields below instead.

Reading The Catalog Fields

Each entry in data carries these Varity fields:

Field	Type	Notes
id	string	Pass this as model in a request
display_name	string	Human label only, never an identifier
maker.key	string	Stable slug of who built the model, for example zai; unreported when unknown
maker.label	string	Human maker label, for example Z.ai
capabilities.input_modalities	array	What a request may carry: text, image, document, audio, video. A model without document rejects a document content part with 400 input_modality_unsupported
capabilities.output_modalities	array	What the model produces: text, embedding, image, audio
capabilities.chat_completions	boolean	true on the chat models only; image, audio, and embedding models report false. Read it per model
capabilities.tool_calls	boolean	true when the model accepts tools / tool_choice. Read it per model; a model reporting false rejects tools
capabilities.max_tools	integer	The most tools one request may carry; above it the request is refused with 400 too_many_tools. Absent means no bound is published
capabilities.max_documents_per_message	integer	The most document parts one message may carry, counted per message (images do not count); above it the request is refused with 400 too_many_documents. Absent means no bound is published
capabilities.streaming	boolean	true when the model accepts stream: true; read it per model
capabilities.streaming_tool_calls	boolean	true when the model streams tool-call deltas; read it per model. Like every proven flag, absent means unknown, not unsupported
capabilities.transcription	boolean	true when POST /v1/audio/transcriptions accepts the model
capabilities.speech	boolean	true when POST /v1/audio/speech accepts the model; speech_formats lists its output formats
context_window	integer	Total tokens, input plus output
max_completion_tokens	integer	Ceiling on generated tokens
privacy	string	Routing privacy class. The live catalog contains both "private" and "anonymized" entries
privacy_routes	array	Route set matching privacy
availability.status	string	"available", "last_known_available", "temporarily_unavailable", "verification_stale", or "unverified", with an observed_at timestamp and, on the last three, a reason
customer_pricing	object	See below
refreshed_at	string	When the gateway last refreshed the entry

Availability Is Withheld, Not Assumed

A model stays listed in /v1/models when its routes are unhealthy — it is not removed — but it stops claiming to be callable. The gateway publishes a route as available only once that route has succeeded since its last failure. A route whose provider refused Varity’s credential, or that failed for any other reason, is excluded from the published projection until a real success clears it; a routing layer’s periodic retry of a failing route is a trial request, not evidence of health, and is deliberately not borrowed by the catalog.

When a model has no publishable route left, its entry changes rather than disappearing:

availability.status	reason	Meaning
available	—	A route succeeded recently. Callable.
last_known_available	—	The entry is being served from the last good observation; observed_at is when that was.
temporarily_unavailable	route_temporarily_unavailable	Routes exist but none is currently publishable. Expect 503 model_unavailable if you call it.
verification_stale	route_verification_stale	The route’s capability certification is too old to stand behind.
unverified	route_unverified	The route was never certified for this capability.

The affected capabilities flags go false alongside the status, so a model that cannot currently serve chat reports chat_completions: false rather than advertising a capability the next request would be refused for.

Caution

This health state is held per gateway replica, in process, and is not shared between them. A replica that has observed a route failing will withhold it while a replica that has not yet seen one may still publish it, so two reads of /v1/models seconds apart can legitimately disagree during a provider incident. Treat availability as a strong hint, not a guarantee, and always handle 503 model_unavailable at call time.

Pricing

Every model in the current catalog returns customer_pricing.status: "configured" with concrete numbers. The schema also allows status: "not_configured", which carries no other fields, so branch on status before reading a rate:

"customer_pricing": {
  "status": "configured",
  "version": "<price-set version>",
  "observed_at": "<ISO-8601 timestamp>",
  "currency": "usd",
  "unit": "per_million_tokens",
  "input": "<number>",
  "output": "<number>",
  "cached_input": "<number>"
}

The values are placeholders here; in the live response the rates are numbers and version and observed_at are strings.

currency is always usd. unit takes one of four values, and it decides which fields carry the rate:

unit	Models	Where the rate is
per_million_tokens	Chat and embedding models	input and output per million tokens; cached_input for cached prompt tokens
per_image	Image generation and editing models	request, a flat price per generated image; input, output, and cached_input are 0
per_million_characters	Speech models	input per million characters of text; output and cached_input are 0
per_audio_second	Transcription models	input per second of audio; output and cached_input are 0

request is present only on per_image entries. cached_input is per model: some models price cached input tokens below input, others at the same rate. Always read unit and currency rather than assuming; the version string identifies the immutable price set the numbers came from and observed_at says when the gateway last observed it. Rates are the current managed rate for the model’s preferred route, not a permanent guarantee: each admitted request snapshots its own price before it runs.

These are the rates for requests sent with an inference key: the provider’s cost plus 5%, with cached-input discounts passed through. Chat and Build in the Developer Portal are charged to the same balance, have no per-action price, and are not priced from this field.

Input and output prices both vary widely across the catalog and change with it, so read customer_pricing per model instead of carrying a range from this page.

Context Windows

Context windows vary per model, and max_completion_tokens is a separate, much smaller ceiling on any single response. A large context window does not imply a large answer, so read both fields per model.

Picking A Model

• Need tool calling? Read capabilities.tool_calls and capabilities.streaming_tool_calls per model, and see Compatibility.
• Need a very large context? Sort the live catalog by context_window. The Grok 4.20 entries (grok-4-20, grok-4-20-multi-agent) have the widest windows in the current catalog, but read the field rather than assuming.
• Cost sensitive? Sort the live catalog by customer_pricing.input within one unit; rates change with the catalog, so do not rely on a named model.

Caution

There is no per-model retrieval route. GET /v1/models/{id} returns 404; fetch the list and filter it client-side.

Next Steps

• AI Gateway Overview: base URL, auth, and the quickstart snippets
• Compatibility: supported and unsupported OpenAI surface