API contract
max_tokens validation across LLM providers
Map max_tokens to provider-specific fields and reject known invalid token caps before upstream calls.
Developers and operators who need predictable output caps without provider-specific client code.
The problem
Output-token fields differ across OpenAI, compatible chat providers, Anthropic, and Gemini, and some providers have known ceilings.
How LLM Proxy helps
LLM Proxy accepts max_tokens and maps it to the selected provider's expected field while validating known Gemini and Claude limits at the request edge.
How it works
- Send max_tokens as a positive integer on supported text endpoints.
- Let the proxy map it to max_output_tokens, max_tokens, or generationConfig.maxOutputTokens.
- Keep Gemini and Claude requests within configured output limits.
- Omit max_tokens to use provider defaults, except Anthropic where the proxy supplies configured limits.
Feature-to-benefit table
| Feature | Why it matters here | Example |
|---|---|---|
| Shared request field | Callers use one max_tokens input across providers. | Provider payload mapping stays backend-owned. |
| Known ceiling checks | Gemini and Claude limits are validated before upstream calls. | Invalid caps return 400. |
| Anthropic requirement handling | The proxy sends required Claude max_tokens from config when omitted. | Clients do not need to know that Anthropic requires the field. |
Use-case examples
Short answer
A caller sets max_tokens=512 for a concise response.
Gemini guard
A Gemini request above 65536 is rejected before provider traffic.
Claude default
A Claude request omits max_tokens and the proxy uses the configured model output limit.
Objections and limitations
- A dash in documentation means the proxy only validates positive values and lets upstream enforce provider-side limits.
- Provider output behavior still depends on the selected upstream model.
- max_tokens affects one request, not tenant-level quota enforcement.
FAQ
What is the main job of max tokens provider limit validation?
LLM Proxy accepts max_tokens and maps it to the selected provider's expected field while validating known Gemini and Claude limits at the request edge.
Who should read this api contract resource?
Developers and operators who need predictable output caps without provider-specific client code.
Does this page claim provider performance or pricing advantages?
No. The supported claim is about LLM Proxy's documented routing, configuration, management, security, usage, and deployment contracts. Provider cost, speed, rankings, and benchmark claims are not made here.
Where should setup details come from?
Use the main README and implementation notes for current command, config, and endpoint details. This page summarizes the workflow without replacing LLM Proxy documentation.
What should I read next?
A closely related resource is Canonical /v2 chat messages API for LLM calls, which covers v2 chat messages API.
Related resources
Use this pattern in LLM Proxy
Start from the canonical API reference, then use the management surface when the workflow needs tenant or provider configuration.
Open API reference