Otari routes requests across multiple providers through one OpenAI-compatible endpoint, with automatic failover when a provider fails. Your app sends one request with one key, and Otari handles provider selection behind it.
This guide sets up two providers, OpenAI and Anthropic, with a fallback chain, using hosted otari.ai.
Otari supports 40+ providers, including OpenAI, Anthropic, Amazon Bedrock, Azure OpenAI, Mistral, Gemini, and local models.
Provider keys vs API keys
Before we get into setup, it helps to separate two similar-sounding concepts:
Provider keys are your upstream credentials, like your OpenAI or Anthropic API key, stored in Otari.
API Keys are workspace-scoped keys your application uses to call Otari.
Your app sends an API Key to Otari. Otari uses the provider key to call the upstream provider. Your app never sees the upstream secret.
What changes with Otari
Without Otari, your app has to know which provider to call, then how to authenticate, what model name to send, and what to do when your request fails. Rinse and repeat for each provider you use.
With Otari, your app sends one request to one endpoint:
Otari resolves the route based on your workspace configuration and can move to the next provider in the chain when a transient upstream failure happens.
Your application talks to Otari once. Otari handles the provider-facing complexity.
What you need
- a hosted `otari.ai` account
- a workspace where you want to route requests
- at least two provider accounts and API keys for this walkthrough
- org admin access, if you want to add provider keys and manage routing policies
This example uses OpenAI and Anthropic because they make the routing story easy to follow. If your team uses Bedrock or Azure OpenAI instead, the flow is the same.
Step 1. Add your provider keys
Start with bring-your-own-provider setup, because that is where multi-provider complexity usually begins.
1. Open Organization Settings → Provider Keys.
2. Under Your provider keys, click Add.
3. Add your OpenAI key.
4. Add your Anthropic key.
5. Set workspace access so both keys are available to the workspace you want to use.
6. If you have multiple keys for the same provider, set the right default.
At this point, Otari knows how to reach both providers without your application needing to store either key.
Step 2. Create a workspace API Key
Next, create the credential your app will actually send to Otari.
1. Open the target workspace.
2. Go to the API Keys tab.
3. Click Generate.
4. Copy the key immediately.
You can export it locally like this:
export OTARI_API_KEY=tk_your_api_key
This is the only credential your application needs to send requests through Otari.
Step 3. Create a routing policy
Now we get to the part that turns “multiple provider access” into “multi-provider routing.”
Switch to the target workspace.
Open Routing.
Click New policy.
Name it something like
production-chat-routing.Turn on Fallback enabled.
Turn on Default policy for this workspace.
Select openai from provider dropdown. Add
gpt-4oas model.Click on “Add Entry”. Add anthropic as provider, and claude-sonnet-4-6 model for entry 2.
Save the policy.
That gives you an ordered fallback chain. Otari will try the first entry first. If the request hits a transient upstream issue, such as a timeout, 5xx response, rate limit, or bad credentials, it can move to the next configured entry automatically.
If your second route should be Bedrock instead of Anthropic, swap entry 2 for the Bedrock model your team uses.
Routing flow
The important part is that your application still makes one request. The fallback chain lives in Otari, not in your app code.
Step 4. Send your first routed request
Once the default routing policy is in place, your app can send a normal OpenAI-compatible request to Otari.
curl https://api.otari.ai/v1/chat/completions \
-H "Authorization: Bearer $OTARI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "openai:gpt-4o",
"messages": [
{
"role": "user",
"content": "Explain why multi-provider routing matters in two short sentences."
}
]
}'
The request format stays simple:
one endpoint
one auth pattern
one request shape
What changes is the control plane behind it. Otari resolves the route for the workspace and applies the policy you configured.
Step 5. Verify the result
After the request succeeds:
Open the same workspace.
Go to the Usage tab.
Confirm the request appears there.
The Usage tab records the request and its cost, so spend across all providers is visible in one place.
Why this matters in practice
Multi-provider routing is not just a nice-to-have abstraction. It changes how you operate AI in production.
You stop hardcoding provider choices into application logic.
You can swap providers without rewriting every caller.
You can keep upstream credentials out of your application code and deployment config.
You can add resilience without building your own retry and fallback layer.
You get one place to manage provider access, request visibility, and spend controls.
That is the real win. Otari lets your application stay focused on product behavior while routing stays in the infrastructure layer where it belongs.
Test the fallback
So far every request has succeeded on the primary, so you haven't seen the fallback do anything yet. You can trigger it on purpose to confirm the chain works.
Temporarily break the primary, then send the same request:
In Organization Settings → Provider Keys, replace your OpenAI key with an invalid one, or revoke it. Leave the Anthropic key valid.
Send the same
openai:gpt-4orequest from Step 4, unchanged.
curl https://api.otari.ai/v1/chat/completions \
-H "Authorization: Bearer $OTARI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "openai:gpt-4o",
"messages": [
{"role": "user", "content": "Explain why multi-provider routing matters in two short sentences."}
]
}'
The request still succeeds, but the response now comes from Anthropic. OpenAI returned a 401, Otari moved to the next entry, and Anthropic served the call. Two fields in the response show what happened:
{
"id": "msg_01CDQMjizxtan5ET2GHEZwGH",
"model": "claude-sonnet-4-6",
"choices": [
{
"index": 0,
"finish_reason": "stop",
"message": {
"role": "assistant",
"content": "Multi-provider routing prevents vendor lock-in ..."
}
}
],
"usage":
{
"prompt_tokens": 20,
"completion_tokens": 51,
"total_tokens": 71
}
}
You requested openai:gpt-4o, but the model came back as the Anthropic model, and the id starts with msg_, Anthropic's response shape, rather than chatcmpl-, OpenAI's. The model field is the simplest thing to check: if it differs from what you requested, the request fell over to the next entry. The exact model string matches whatever you configured as entry 2.
The Usage tab confirms it durably: the call is recorded against Anthropic, not OpenAI.
This is different from naming Anthropic in the request directly. Here you asked for openai:gpt-4o and the policy's fallback chose Anthropic because the primary failed. The model in the request is the entry point, not a guarantee of which provider serves it.
When you're done, restore your real OpenAI key so the primary route works again.
A good first production pattern
If you want a practical starting point, use this setup:
OpenAI as the primary route for your main chat workload.
Anthropic as the fallback route for transient failures.
One default routing policy per workspace.
One workspace API Key per environment or application.
That is enough to move from a single-provider setup to a real multi-provider architecture without changing your application contract.
Make your first routed request
If you already have provider keys for two vendors, you are closer than you think.
Add the keys once, create one workspace API Key, define one default routing policy, and send one request through otari.ai. That is the moment Otari starts paying off: your app keeps one interface, while your infrastructure becomes more flexible, more resilient, and easier to maintain.