API contract
Canonical /v2 chat messages API for LLM calls
Send ordered system, user, and assistant messages through one v2 endpoint before provider routing.
Developers who want a stable chat transcript contract instead of provider-specific payloads.
The problem
Chat transcript callers can end up building OpenAI, Anthropic, Gemini, and compatible-provider request bodies separately, with different message rules in each client.
How LLM Proxy helps
LLM Proxy exposes /v2 as the canonical chat endpoint. Its artifact-defined request fields are messages, model, web_search, max_tokens, reasoning_effort, and the proxy maps that shared request to the selected provider.
How it works
- Send POST /v2 with messages[] and the tenant key query parameter.
- Use system role messages for instructions rather than a body system_prompt.
- Provide order values only when every submitted message has a unique non-negative order.
- Let omitted model resolve to the tenant or selected-provider default.
- Omit reasoning_effort to retain the tenant default, or send a nonblank value declared by the exact resolved route.
Feature-to-benefit table
| Feature | Why it matters here | Example |
|---|---|---|
| Messages-only input | The canonical endpoint rejects prompt and body system_prompt fields. | Ambiguous input fails before an upstream call. |
| Order support | Messages can be sorted by explicit order when callers cannot rely on array position. | Event-sourced transcripts can still route deterministically. |
| Route-bound reasoning effort | A caller can override the tenant default only with a nonblank value declared by the exact resolved provider/model route. | Blank, null, and unsupported values fail with 400 before an upstream call. |
| Provider mapping | The proxy maps shared messages into OpenAI, Claude, Gemini, or compatible provider shapes. | Client code does not need a provider SDK. |
Use-case examples
Chat transcript
A support tool sends system, user, and assistant messages to /v2 with provider omitted.
Provider experiment
The same /v2 body is sent with provider=anthropic for a Claude trial.
Ordered events
A caller sends order values from persisted event IDs so the proxy sorts the transcript.
Objections and limitations
- POST /v2 requires at least one user message.
- Unsupported roles, duplicate order values, and mixed prompt fields return 400.
- Reasoning effort is not a global option list; a value unsupported by the resolved route returns 400.
- Server-injected tenant default system prompts are sent upstream but not echoed in response metadata.
FAQ
What is the main job of v2 chat messages API?
LLM Proxy exposes /v2 as the canonical chat endpoint. Its artifact-defined request fields are messages, model, web_search, max_tokens, reasoning_effort, and the proxy maps that shared request to the selected provider.
Who should read this api contract resource?
Developers who want a stable chat transcript contract instead of provider-specific payloads.
Does this page claim provider performance or pricing advantages?
No. The supported claim is about LLM Proxy's documented routing, configuration, management, security, usage, and deployment contracts. Provider cost, speed, rankings, and benchmark claims are not made here.
Where should setup details come from?
Use the main README and implementation notes for current command, config, and endpoint details. This page summarizes the workflow without replacing LLM Proxy documentation.
What should I read next?
A closely related resource is Large prompt JSON POST for LLM requests, which covers large prompt JSON POST.
Related resources
Use this pattern in LLM Proxy
Start from the canonical API reference, then use the management surface when the workflow needs tenant or provider configuration.
Open API reference