Security
Tenant-secret AI gateway for internal applications
Give client apps one llm-proxy tenant secret while the service owns provider credentials and routing.
Teams building internal apps that need a single guarded AI service boundary.
The problem
When every app gets its own provider credential and routing rules, access control becomes difficult to reason about and providers become embedded in product code.
How LLM Proxy helps
LLM Proxy keeps public proxy requests authenticated by key=<tenant secret>, then applies the tenant's configured defaults and server-side provider credentials.
How it works
- Create a tenant in static config or through the management UI.
- Generate or configure a tenant secret for the client.
- Use the same key parameter across GET, POST, /v2, and /dictate.
- Replace the secret or delete its owning managed tenant when access should change.
Feature-to-benefit table
| Feature | Why it matters here | Example |
|---|---|---|
| Single credential surface | Apps call one proxy secret instead of many provider keys. | The same secret can protect text and dictation traffic. |
| Tenant defaults | Omitted provider and model values resolve through tenant defaults. | A caller can stay simple until it needs a per-request override. |
| 403 boundary | Missing, invalid, or replaced tenant secrets return forbidden responses. | Callers do not receive tenant discovery details. |
Use-case examples
Shared app platform
A platform team provisions tenant secrets for several internal apps while keeping provider credentials central.
Key replacement
Replacing a generated secret immediately stops the prior value from authenticating future proxy calls.
Dictation plus text
The same tenant-secret model can protect voice transcription and LLM response generation.
Objections and limitations
- Tenant secrets should not be placed in public client code without appropriate controls.
- Management mode uses database state for generated secrets; persistence must be configured.
- This is not a full identity provider; it is the proxy authentication boundary.
FAQ
What is the main job of tenant secret AI gateway?
LLM Proxy keeps public proxy requests authenticated by key=<tenant secret>, then applies the tenant's configured defaults and server-side provider credentials.
Who should read this security resource?
Teams building internal apps that need a single guarded AI service boundary.
Does this page claim provider performance or pricing advantages?
No. The supported claim is about LLM Proxy's documented routing, configuration, management, security, usage, and deployment contracts. Provider cost, speed, rankings, and benchmark claims are not made here.
Where should setup details come from?
Use the main README and implementation notes for current command, config, and endpoint details. This page summarizes the workflow without replacing LLM Proxy documentation.
What should I read next?
A closely related resource is Authenticate an LLM Proxy client with a tenant secret, which covers LLM Proxy client authentication.
Related resources
Use this pattern in LLM Proxy
Start from the canonical API reference, then use the management surface when the workflow needs tenant or provider configuration.
Open API reference