Hybrid Mode

When Otari runs in hybrid mode (OTARI_AI_TOKEN is set), it delegates per-request authorization and provider-credential resolution to a peer platform service over HTTP. This document describes the wire contract Otari expects from that peer.

The reference peer is the otari.ai platform, but any service that implements this contract can stand in.

Otari calls these endpoints, all rooted at the configured platform base URL:

{base} means Otari platform base_url setting. Otari concatenates literally. The peer service is responsible for including any API-version prefix it exposes its own routes under. For the reference otari deployment that prefix is /api/v1, so the base URL is http://backend:8000/api/v1 and Otari ends up POSTing to http://backend:8000/api/v1/gateway/provider-keys/resolve.

Every endpoint requires X-Gateway-Token: <gw_...> in the request headers. This proves the caller is an Otari instance configured against this platform deployment. The three resolve endpoints additionally require X-User-Token: <tk_...>, which is the workspace API token forwarded opaquely from the end user's Authorization: Bearer ... header. The usage endpoint sends only the gateway token.

This document describes only the fields Otari actually reads. Every response shape below is open for extension: a peer may return additional fields, and Otari ignores any it does not recognise. Consumers of this contract MUST do the same (ignore unknown fields) so the platform can add fields without breaking older gateways.

For example, the otari.ai resolve response also carries workspace_id, organization_id, provider_key_id, and allowed_models. Otari does not read these today, so they are intentionally absent from the shapes documented here.

When the operator points Otari's web-search backend at the platform (OTARI_WEB_SEARCH_URL under base_url), Otari also sends X-Gateway-Token on its search queries (GET {base}/gateway/web-search/search) so a platform-hosted search endpoint can authenticate the gateway. The token is sent only when that URL shares the platform origin (scheme/host/port, under the base path); it is never sent to a standalone or third-party search backend.

Otari iterates attempts in order. On a retryable failure it moves to the next entry; on success it stops. The attempt_id of the entry that ultimately succeeded (or the last one tried, on total failure) is what Otari echoes back via X-Correlation-ID and reports through /gateway/usage.

request_id groups every attempt_id from the same resolve call so the platform can attribute spend, render trace timelines, and emit fallback events. Otari also surfaces it as the X-Otari-Request-ID response header.

fallback_enabled is informational — set by the platform when its routing policy actually allows fallback (i.e. the policy has multiple enabled entries and fallback_enabled = true). Otari uses len(attempts) > 1 for its own behaviour.

attempts MUST contain at least one entry. An empty list is treated as a platform bug and surfaced as 502 Bad Gateway.

Otari also accepts a flat payload:

Otari maps this onto a single-attempt route (attempts = [{...}], fallback_enabled = false) and behaves as it always has — no retry loop, errors propagate to the client. New platform implementations should prefer the multi-attempt shape.

Called only when a request references one or more workspace-scoped MCP server ids (a hybrid-only feature). Otari swaps those ids for the inline server configs it needs to open the connections.

Otari reads name, url, authorization_token, purpose_hint, and allowed_tools off each entry in servers; a missing servers key is treated as an empty list. The same URL-safety rules as inline MCP configs apply once the configs are resolved (SSRF guard, no bearer token over cleartext http://).

Called only when a request uses the otari_web_search tool. The platform owns the per-workspace web-search policy: whether it is enabled at all, plus the workspace-default limits and filters.

The request body is empty; the workspace is identified by X-User-Token.

If enabled is false, Otari rejects the request with 403. The remaining fields are workspace defaults that apply only where the request did not supply its own value: max_results, allowed_domains, blocked_domains, and purpose_hint fill in when the per-request tool entry omits them (an empty list or empty string reads as "no preference" and does not clear the workspace value), and provider_options is shallow-merged with per-request keys winning. provider is informational: the active web-search backend is configured on the gateway itself via OTARI_WEB_SEARCH_URL, so Otari does not switch backends based on this field.

The resolve endpoints share the timeout (PLATFORM_RESOLVE_TIMEOUT_MS) and token headers with provider-keys/resolve. Their exact response shapes will become the contract of record once the consumer-side fixtures land (#146); until then this document is authoritative.

After every attempt — successful or failed — Otari sends:

session_label is an optional caller-supplied label for cost attribution (per run, experiment, or conversation). A caller sets it on the request body (session_label on the chat/messages/responses request); Otari strips it before the upstream provider call and forwards it here so the platform can attribute the attempt's spend to that session without the caller standing up OpenTelemetry. It is trimmed and omitted when blank; Otari caps it at 255 characters at the request boundary so the platform never has to truncate. All attempts of one request carry the same label.

user is not used for cost attribution in hybrid mode. The OpenAI-standard user field is stripped before the upstream call and is not forwarded on the usage report, so it does not segment spend here. Callers who want per-run cost breakdown must use session_label.

cache_read_tokens and cache_write_tokens are additive fields carrying the provider cached-token counts (default 0 when a provider reports none). Their inclusion convention differs by provider, so the platform must price them with that in mind:

The platform must accept these additive keys with lenient parsing; a handler that rejects unknown fields would 422 the report (a non-retryable status), silently dropping it. See companion issue mozilla-ai/otari-ai#1168.

A multi-attempt request that iterates two attempts produces two usage reports — one per attempt — sharing the same request_id (recoverable via the original resolve response). The platform is responsible for correlating them.

error_class is a short tag describing why the attempt was abandoned:

The field is omitted entirely when Otari can't classify the failure back to an exception — this happens with mid-stream errors surfaced via the SSE channel, where only an error string is available. Treat a missing error_class as "uncategorised error" when aggregating.

The usage endpoint is called as a background task on Otari side. It retries on transient failures (timeout, network error, 5xx) up to PLATFORM_USAGE_MAX_RETRIES times with exponential backoff (0.25s, 0.5s, 1s). It does not retry on 401, 404, 409, 422 — those are treated as terminal client errors.

Streaming requests (stream: true) iterate attempts just like non-streaming requests, with one structural difference: Otari can only fall through before any bytes have been flushed to the client. Once an attempt yields its first chunk, Otari commits to that attempt; any further error propagates to the SSE channel as today.

The mechanism is a per-attempt first-chunk gate. For each attempt:

Latency contract: zero added latency in the success case — the first chunk is held only for the microseconds it takes to call the SSE response builder. In the failure case, each abandoned attempt costs at most first_chunk_timeout_seconds.

What this catches: auth errors (401/403), rate-limits (429), upstream 5xx, connection failures, hung connections, "stream opens but errors before yielding."

What this doesn't catch: errors that arrive after the first chunk has flushed (mid-stream connection drops, refusal messages embedded in normal content chunks). These are out of reach without either prefix-buffering (which would add visible latency on every request) or a client-cooperative restart event (which would break OpenAI SDK compatibility).

Mid-stream failover is not currently planned. If a future client SDK starts honouring a custom restart event, it could be added behind that capability flag.