Twelve modules, one panel.
Serving, governance, acceleration, and understanding — each documented, each observable, each manageable without a restart.
Serve every request.
The compatible surface, the routing decision, and the failure path that holds.
- OpenAI-compatible APIPOST /v1/chat/completions, streaming and buffered, plus /v1/models. An SDK swap, not a rewrite — extra fields ride under synapass and OpenAI clients ignore them.02 surfacesinfer + admin
- Multi-provider routingpriority, weighted, lowest_cost, lowest_latency, highest_quality — over a registry with real context windows, capabilities, and prices. Computed per request, recorded always.05 patternsper request
- Fallback chainsOrdered failover with bounded retry and deterministic jitter. Unhealthy providers leave rotation via circuit breaker; recovered ones return. A started stream is reported, never appended to.boundedretry + jitter
Govern every request.
Policy, spend, and identity — enforced before the call, explained after it.
- Policy engineMatch on model, tenant, key, task, size, region, sensitivity, and endpoint. Resolved by specificity, so a specific rule always beats a broad one. Denials name the rule.07 dimsmatched
- Cost controlPer-request ceilings checked against projected cost before the call, token and request rate limits, daily and monthly budgets. Runaway loops meet a limit, not an invoice.pre-callceilings
- Tenants & keysTenants isolate routing, cache, and spend. Keys are SHA-256 digests returned exactly once, scoped by method and path, revocable with immediate effect.01 returnof plaintext
Accelerate every request.
Answer from cache, execute tools under bounds, keep providers honest.
- CachingExact, prefix, and semantic tiers over Redis — tenant-isolated by construction, invalidatable by scope with an audit trail. Hit rates and bypass reasons in the dashboard.03 tiersisolated
- ToolsClient-executed by default; bounded gateway-side execution behind a policy when you want it, with operator-owned limits and a durable step trace per run.boundedstep traces
- Providers & modelsAdd, test, sync, probe, kill, and revive providers. Discover remote models while preserving local pricing, aliases, and disables. Aliases keep app code stable.liveno restarts
Understand every request.
Lineage, analytics, and audit — the proof behind every decision.
- Request lineageTask classification, policy verdict, cache decision, prompt-shaping plan, and routing reason on every request — readable back via /explain. Errors carry the same block.09 stepsexplained
- AnalyticsThroughput, latency percentiles, spend over time, cache performance, and provider and model quality with explanations — in the dashboard and ClickHouse for your own queries.p50–p99percentiles
- Observability & auditPrometheus metrics, OTLP traces, JSON logs, Grafana dashboards and alert rules out of the box. Every control-plane mutation is audit-logged with before and after values.04 signalsshipped
Read the module you need, end to end.
Every module above has a full docs page with steps, code, and edge cases.