Provider routing
OpenAI-compatible provider gateway
Route Meta Muse Spark, DeepSeek, Qwen through Alibaba Cloud, Kimi, MiniMax, SiliconFlow, Z.AI, and Grok text calls through one compatible adapter.
Teams adopting OpenAI-compatible chat providers without rewriting every caller.
The problem
OpenAI-compatible providers share a broad shape but still need different base URLs, keys, defaults, and availability rules.
How LLM Proxy helps
LLM Proxy uses one compatible chat adapter. Provider transports and offerings stay in providers.yml, while tenant values stay in management storage.
How it works
- Declare provider transports and offerings in providers.yml. Save credentials in an account connection and connect the tenant.
- Use provider selectors such as meta, deepseek, dashscope, moonshot, minimax, siliconflow, zai, or xai.
- Send GET, compatibility POST, or canonical /v2 requests.
- Let omitted model use the selected provider's configured default.
Feature-to-benefit table
| Feature | Why it matters here | Example |
|---|---|---|
| Shared adapter | Compatible chat providers use one proxy integration pattern. | Provider-specific logic stays centralized. |
| Provider aliases | Some providers expose aliases such as qwen or kimi. | Callers can use documented selectors. |
| Unconfigured-provider behavior | A selected provider without a saved tenant key returns 409 provider_not_configured before dispatch. | Operators can add provider credentials without changing service config. |
Use-case examples
Meta Muse route
A caller sends provider=meta and model=muse-spark-1.2 through Chat Completions.
Qwen alias
A caller uses provider=qwen for Alibaba Cloud routing.
MiniMax route
A caller can select any configured MiniMax M2 route, such as provider=minimax and model=minimax-m2.7-highspeed, through the shared Chat Completions adapter.
xAI route
A Grok text request uses the OpenAI-compatible chat adapter behind provider=xai.
Objections and limitations
- Meta support is text-only; the proxy does not expose Meta dictation, web search, tools, multimodal inputs, or a Responses fallback.
- Each provider needs a saved tenant key before serving tenant traffic.
- Provider endpoints must stay explicit in the provider catalog.
FAQ
What is the main job of OpenAI-compatible provider gateway?
LLM Proxy uses one compatible chat adapter. Provider transports and offerings stay in providers.yml, while tenant values stay in management storage.
Who should read this provider routing resource?
Teams adopting OpenAI-compatible chat providers without rewriting every caller.
Does this page claim provider performance or pricing advantages?
No. The supported claim is about LLM Proxy's documented routing, configuration, management, security, usage, and deployment contracts. Provider cost, speed, rankings, and benchmark claims are not made here.
Where should setup details come from?
Use the main README and implementation notes for current command, config, and endpoint details. This page summarizes the workflow without replacing LLM Proxy documentation.
What should I read next?
A closely related resource is Integrate once through one multi-provider LLM proxy, which covers multi-provider LLM proxy.
Related resources
Use this pattern in LLM Proxy
Start from the canonical API reference, then use the management surface when the workflow needs tenant or provider configuration.
Open API reference