Otari
Otari is an OpenAI-compatible LLM gateway. It routes to 40+ providers, manages API keys, enforces budgets, and records usage. It is also the gateway at the heart of otari.ai.
New here?
Start with the Quickstart.
Browse the docs
The docs are grouped by what you are trying to do.
Start here
Quickstart: get running and make your first request.
For operators
Running and managing a gateway.
Deployment: Docker, Railway, and optional services.
Configuration: configuration sources, precedence, and common settings.
Admin dashboard: sign-in, setup, and management surfaces.
Access control: identities, organizations, workspaces, keys, and budgets.
Workload identity: a backend on Google Cloud authenticates with its platform identity token instead of a key's secret.
Models: selectors, providers, discovery, aliases, and capabilities.
Hosted providers: credentials the deployment pays for, and how requests on them are billed.
Provider endpoints: model servers a workspace or a user brings, reached by name and kept out of budgets.
Routing policies: failover, conditions, weighted, priority and learned routing, and mandatory guardrails.
Health checks: a scheduled check of every model the deployment serves, and email when one stops answering.
OpenAI provider guide: configure OpenAI and route your first request through Otari.
Gemini provider guide: configure Gemini and route your first request through Otari.
For integrators
Calling the gateway from your own code.
API reference: authentication, endpoint availability, and links to generated schemas.
Built-in tools: sandboxed code execution and web search Otari runs itself.
MCP: connect MCP servers to chat, messages, and responses requests, or drive them from your own application through the caller-orchestrated endpoints.
Files: file uploads and document understanding for local models.
Inference Guardrails: request-level checks like prompt-injection detection.
Agent Guardrails: repository-owned rules checked against what a coding agent actually did to the working tree.
otari hookruns beside Claude Code or Codex and evaluates them locally, with no server and no credential.Agent Guardrails reference: every gate type and field, and what each harness can see at each event.
Use with Claude Code: point the Claude Code CLI at Otari.
Use with Codex: route the Codex CLI through Otari over the Responses API, or import its usage without routing.
Use with Junie: point the Junie CLI or JetBrains AI Assistant at Otari.
Use with opencode: point the opencode CLI at Otari.
Use with Pydantic AI, Use with LangChain and Use with LangGraph: point an agent framework's model at Otari, with tools, structured output and streaming.
Use with a ChatGPT subscription: route Otari at ChatGPT Plus/Pro models through a local Codex-OAuth proxy.
Agent traces: see each agent session as turns, LLM calls and tool calls, with their timing and cost.
Importing external usage: bring subscription-backed usage (Claude Code, Codex, any OTLP app) into your analytics.
SDK compatibility: how the language SDKs are released and which SDK version works with which Otari version.
For platform builders
Code-execution protocol: the Otari/sandbox contract, for building a code-execution backend that Otari dispatches to. Its machine-readable form is
public/code-execution-openapi.yaml.
For contributors
Architecture: one process over one database, and the extension seam (ports, adapters, and capability lines) that mark what Otari's core ships versus what an overlay can add.
Backend domains: what each backend domain owns, and the layer shape each domain moves toward.
Load test: compares two builds under load on one machine, route by route, and fails when the second got slower or runs more database statements per request. CI runs it on a PR against its base.
Last updated
Was this helpful?
