Providers

A provider is one upstream inference service. Synapass talks to it through an adapter, knows what it costs and what it can do, and routes requests to it according to policy. This page covers adding providers, credentials, kinds, testing and day-to-day management.

Provider kinds#

Sampling controls are forwarded only to kinds that implement them. Sending top_k to a provider that rejects it turns a working request into a 400.

Adding a provider#

Through the dashboard#

Open Providers → Add. The form takes a name, kind, base URL and credential source. Tick Sync models to discover the provider's remote models in the same call; the per-row Sync button re-runs discovery later.

Through the API#

curl -s $GATEWAY/admin/v1/providers \
  -H "Authorization: Bearer $SYNAPASS_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "openai-prod",
    "kind": "openai",
    "base_url": "https://api.openai.com/v1",
    "api_key_env": "OPENAI_API_KEY",
    "sync_models": true
  }' | jq

name is immutable. To rename, delete and recreate.

Discovery inserts every missing remote name as an active, api-managed row inheriting the provider's environment. Existing rows are never modified, so your pricing, aliases, priorities and disables survive a re-sync. Delete a row to drop a model permanently.

A discovered model arrives knowing its name and nothing else — the OpenAI /v1/models response carries no capability metadata — so it inherits the provider kind's defaults until someone says otherwise. Two ways to say otherwise, in order of preference:

1. Declare it when you add the model, or PATCH the row later. What you write is final: nothing automatic ever overwrites a declared list.
2. Read it from the provider (POST …/sync-models, or tick Sync models on create). Routers that publish it — LiteLLM-style /model/info with supports_function_calling and friends — have it recorded per model for free.
3. Ask the model (POST …/detect-capabilities). It sends each silent model minimal probes — a one-token completion carrying tools, an event-stream attempt, then JSON modes — and records only what a successful call proves. A 400 that names the feature means it cannot do it; auth failures, rate limits and 5xx mean nothing was decided and the row is left alone. Provenance lands in the row metadata as capabilities_source: provider | probed | declared. Chat is recorded whenever anything succeeds, because every probe is itself a chat completion — a proven list without it would unroute the model from plain requests.

Detection spends real upstream tokens (four tiny calls per model by default), so the automatic post-discovery run is off: set detection.enabled: true in the config, or call the endpoint with an explicit {"models": [...]}. Bounds (max_models_per_run, concurrency, timeout_per_model) keep one click from becoming a bill, and an explicit model list scopes it further. Vision is never probed — it needs an image payload, a different cost class from a one-token text probe.

Credentials#

Two sources, with fixed precedence.

The environment binding wins when both are present. Setting both is allowed and the detail view shows both facts, so the precedence is documented rather than silent.

curl -s -X PUT $GATEWAY/admin/v1/providers/$ID/credential \
  -H "Authorization: Bearer $SYNAPASS_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"secret":"sk-…","name":"primary"}' | jq

Stored credentials are sealed with AES-256-GCM in provider_credentials, one active credential per provider. Reads return metadata only:

curl -s $GATEWAY/admin/v1/providers/$ID/credential -H "Authorization: Bearer $SYNAPASS_ADMIN_KEY" | jq
# → {"has_credential": true, "credential": {"name": "primary", "key_version": 3}}

The plaintext never appears in a response, a log, an audit event or the UI. The first write audits create; overwriting audits rotate.

Key material#

Rotating SYNAPASS_ADMIN_KEY without setting SYNAPASS_CREDENTIALS_KEY orphans stored credentials — re-save them after a rotation.

Resolution timing#

Credentials are resolved once per catalogue refresh, never on the request path. A provider with neither binding constructs no adapter and is excluded from routing rather than failing requests.

Models#

A model is a servable name, its context window, its price and its capabilities.

curl -s $GATEWAY/admin/v1/models \
  -H "Authorization: Bearer $SYNAPASS_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "provider_id": "'"$PROVIDER_ID"'",
    "name": "gpt-4o-mini",
    "context_window": 128000,
    "max_output_tokens": 16384,
    "input_cost_per_million": 0.15,
    "output_cost_per_million": 0.6,
    "capabilities": ["chat", "streaming", "tools", "json_mode"]
  }' | jq

name plus provider is the natural key, so renaming or moving a model means delete and recreate. An omitted priority defaults to 100, matching the database default, so you get the same row the bootstrapper would write.

Capabilities the router actually checks:

Required capabilities are derived from the request body, not claimed by the client. A caller cannot declare that it needs vision while sending an image the selected model cannot read.

Testing#

Three checks, each persisted with latency and status:

curl -s -X POST $GATEWAY/admin/v1/providers/$ID/test \
  -H "Authorization: Bearer $SYNAPASS_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"checks":["connectivity","models","sample"],"model":"gpt-4o-mini","prompt":"say ok"}' | jq

checks defaults to all three; an unknown check name is a 400. Checks never abort the run early — each persists to provider_test_results and the run audits test once with a summary. History is GET …/tests?limit= (default 50).

The handler reloads the catalogue first, so a provider created moments ago already has an adapter when tested.

Health#

Every provider carries a circuit breaker fed by two sources: active probes on health_check_interval, and passive observation of live traffic.

consecutive failures ≥ 3   → degraded   (still eligible, deprioritized)
consecutive failures ≥ 9   → unhealthy  (excluded from routing)
rolling error rate > 50%   → degraded

Marking a provider unhealthy too eagerly offloads its traffic onto its peers, which is worse than occasionally routing to one that is merely struggling, so the thresholds are asymmetric on purpose. A successful probe does not by itself close an open circuit — a probe that succeeds while live traffic fails would flap.

Run an on-demand probe:

curl -s -X POST $GATEWAY/admin/v1/providers/$ID/probe -H "Authorization: Bearer $SYNAPASS_ADMIN_KEY" | jq

Inspect current state:

curl -s "$GATEWAY/admin/v1/providers/health?provider_id=$ID" -H "Authorization: Bearer $SYNAPASS_ADMIN_KEY" | jq

Managing providers#

PATCH {"status":"disabled"} takes a provider out of rotation; re-enabling audits enable rather than a plain update, so the audit log answers who turned it back on.

Ownership#

Every provider, model and policy carries managed_by: bootstrap for rows owned by the configuration seeder, api for rows created or edited through this surface. The bootstrapper never overwrites an api-managed row on restart, which is what keeps a dashboard edit from being silently reverted by the next deploy. To hand a row back to the config file, delete it through the API and let the seeder recreate it.

Common problems#

Related: Routing · API reference · Dashboard · Troubleshooting · Back to README