Varity AI Gateway

Varity TeamCore Contributors

The Varity AI Gateway is an OpenAI-compatible inference API. It speaks the same chat-completions wire format as the OpenAI API, so every OpenAI SDK, framework adapter, and agent runtime already knows how to talk to it. For chat, streaming, tool calls, embeddings, image generation and editing, speech, and transcription, the integration is a one-line base-URL swap.

https://ai.varity.app/v1

https://ai.varity.app is the canonical AI Gateway host; the older varity.app/v1 alias is described on Endpoints.

Chat completions are the core, and the same host serves OpenAI-compatible embeddings, image generation and editing, transcription, and speech on the models whose capabilities advertise them.

Fastest Path

Set two variables and send a request. This is the whole integration.

curl

curl https://ai.varity.app/v1/chat/completions \
  -H "Authorization: Bearer $VARITY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5-2",
    "messages": [
      { "role": "user", "content": "Say hello in one sentence." }
    ]
  }'

Python

# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://ai.varity.app/v1",
    api_key=os.environ["VARITY_API_KEY"],
)

response = client.chat.completions.create(
    model="glm-5-2",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)

print(response.choices[0].message.content)

Node

// npm install openai
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://ai.varity.app/v1",
  apiKey: process.env.VARITY_API_KEY,
});

const response = await client.chat.completions.create({
  model: "glm-5-2",
  messages: [{ role: "user", content: "Say hello in one sentence." }],
});

console.log(response.choices[0].message.content);

base_url / baseURL is the only line that differs from an OpenAI setup. Model ids come from the Varity catalog, not OpenAI’s. See Models for the full list.

Confirm The Gateway Before You Have A Key

GET /v1/models is public. Run it right now, with no credential, and you get a 200 plus the live catalog:

curl https://ai.varity.app/v1/models

{
  "object": "list",
  "privacy_scope": "anonymized",
  "data": [
    {
      "id": "glm-5-2",
      "display_name": "GLM 5.2",
      "object": "model",
      "owned_by": "varity",
      "maker": { "key": "zai", "label": "Z.ai" },
      "capabilities": {
        "input_modalities": ["text", "document"],
        "output_modalities": ["text"],
        "chat_completions": true,
        "tool_calls": true,
        "streaming": true,
        "streaming_tool_calls": true
      },
      "context_window": 1000000,
      "max_completion_tokens": 131072,
      "privacy": "private",
      "privacy_routes": ["private"],
      "availability": { "status": "available" },
      "customer_pricing": {
        "status": "configured",
        "currency": "usd",
        "unit": "per_million_tokens",
        "input": "<live rate>",
        "output": "<live rate>",
        "cached_input": "<live rate>"
      }
    }
  ]
}

The entry above is trimmed, and the rates are placeholders: input, output, and cached_input are numbers in the real response. Read prices from GET /v1/models or the Models tab on the portal’s AI Gateway page, never from this page.

The gateway also exposes unauthenticated health and readiness probes on the canonical host; their response shapes are on Endpoints.

Authentication

Inference requests require a Varity API key sent as a bearer credential:

Authorization: Bearer $VARITY_API_KEY

Create keys on the Developer Portal’s AI Gateway page, under the Inference keys tab, or with POST /v1/keys from a signed-in portal session. Key plaintext is shown once at creation and is never returned again; the list identifies each key by its key id (vki_…). The same key format is used across the Varity public API. A revoked key stops working within 60 seconds. See Billing and Keys.

Inference is prepaid: add funds on the Gateway billing tab before your first request. Each request is charged according to the live model catalog; funding terms come from the live OpenAPI commercial-contract block.

Auth is enforced on every inference route. A request with no Authorization header returns 401:

{
  "error": {
    "message": "A bearer credential is required.",
    "type": "authentication_error",
    "code": "authentication_required",
    "param": null,
    "retryable": false,
    "request_id": "516c24ef-ba5a-4e3d-8f32-0a609ae5d298"
  }
}

A malformed, revoked, or expired key also returns 401 with code: "authentication_required"; only the message differs: The provided credential is invalid or expired. Treat any 401 as “send a valid key” rather than branching on the message text.

Note

Authentication is checked before the request body is validated. An unauthenticated request naming a nonexistent model still returns the authentication error, not a model error.

Streaming

Set stream: true for server-sent events, exactly as with OpenAI. Read capabilities.streaming per model rather than assuming it is universal.

curl

curl https://ai.varity.app/v1/chat/completions \
  -H "Authorization: Bearer $VARITY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5-2",
    "stream": true,
    "messages": [
      { "role": "user", "content": "Count to five." }
    ]
  }'

Python

stream = client.chat.completions.create(
    model="glm-5-2",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

Node

const stream = await client.chat.completions.create({
  model: "glm-5-2",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Streaming tool calls are a narrower capability than streaming text, and are only supported on some models. Check capabilities.streaming_tool_calls before you combine stream: true with tools. See Compatibility.

Errors

Errors use the OpenAI error envelope, with two Varity additions: retryable and request_id. Quote request_id in any support request. The envelope, the x-request-id header, and every error code are on Compatibility.

Rate Limits

Inference is bounded by a requests-per-minute budget and a concurrency bound, both counted across your whole account. Crossing either returns 429 before any model provider is called. The standard-tier numbers, how to read your own, the headers each refusal returns, and the body-size ceilings are on AI Gateway Rate Limits.

Next Steps

• Endpoints: one minimal request for every route
• Billing and Keys: charges, live funding terms, top-ups, key create and revoke, usage
• Playground: try models in the Developer Portal before writing code
• Models: the full live catalog with makers, context windows, limits, and prices
• Rate Limits: budgets, concurrency, response headers, and the 429 bodies
• Compatibility: exactly what the OpenAI surface includes and excludes, errors, and privacy
• Public API Reference: the /api platform contract for deploys and operations