Security
Keep provider API keys server-side for LLM apps
Use tenant secrets for clients while upstream provider credentials stay on the LLM Proxy server.
Internal-tool builders who need AI calls without distributing raw upstream keys.
The problem
Client apps, notebooks, browser utilities, and scripts can drift into storing raw OpenAI, Anthropic, Gemini, or xAI keys. Once those keys leave the backend, rotation and audit work become harder.
How LLM Proxy helps
LLM Proxy rejects upstream-provider-key-like fields on public proxy requests and reads provider credentials only from server-side config or authenticated management storage.
How it works
- Load provider credentials through config.yml or save them in the authenticated management UI.
- Give clients only an llm-proxy tenant secret.
- Route text and dictation requests with key=<tenant secret>.
- Rotate provider keys in the backend without changing every client integration.
Feature-to-benefit table
| Feature | Why it matters here | Example |
|---|---|---|
| Tenant secret authentication | Clients authenticate to the proxy, not directly to provider APIs. | A browser app can call the proxy without handling an OpenAI key. |
| Provider-key rejection | Public endpoints reject provider key fields in query, JSON, and multipart input. | A mistaken api_key field fails before the upstream call. |
| Managed provider storage | Signed-in users can persist provider keys through authenticated management endpoints. | Mutation responses return masked status; raw retrieval is a separate owner-authenticated reveal action. |
Use-case examples
Browser dashboard
A static Pages app can show copyable proxy examples without embedding upstream keys.
Notebook access
A data notebook uses a tenant secret while provider credentials remain in the service boundary.
Provider rotation
Ops updates the provider key once in backend state instead of touching each app.
Objections and limitations
- A tenant secret is still sensitive and should be handled as an application credential.
- Encrypted-at-rest managed provider keys are not a zero-knowledge guarantee.
- Network controls are still recommended before exposing the service broadly.
FAQ
What is the main job of server-side provider API keys?
LLM Proxy rejects upstream-provider-key-like fields on public proxy requests and reads provider credentials only from server-side config or authenticated management storage.
Who should read this security resource?
Internal-tool builders who need AI calls without distributing raw upstream keys.
Does this page claim provider performance or pricing advantages?
No. The supported claim is about LLM Proxy's documented routing, configuration, management, security, usage, and deployment contracts. Provider cost, speed, rankings, and benchmark claims are not made here.
Where should setup details come from?
Use the main README and implementation notes for current command, config, and endpoint details. This page summarizes the workflow without replacing LLM Proxy documentation.
What should I read next?
A closely related resource is Authenticate an LLM Proxy client with a tenant secret, which covers LLM Proxy client authentication.
Related resources
Use this pattern in LLM Proxy
Start from the canonical API reference, then use the management surface when the workflow needs tenant or provider configuration.
Open API reference