API contract
Large prompt JSON POST for LLM requests
Use JSON POST bodies when prompts are too large or structured for query-string requests.
Developers moving from quick prompt calls to large documents or generated prompt bodies.
The problem
GET query strings are convenient for small prompts, but large or non-ASCII prompts need a request body and clear size validation.
How LLM Proxy helps
LLM Proxy supports compatibility POST / with JSON prompt or messages input and canonical POST /v2 for messages-only chat transcripts.
How it works
- Choose POST / for a prompt field or compatibility messages[] body.
- Choose POST /v2 for the canonical messages-only contract.
- Keep key and provider in query parameters.
- Let max_prompt_bytes enforce request size before provider routing.
Feature-to-benefit table
| Feature | Why it matters here | Example |
|---|---|---|
| JSON body support | Large prompts can move out of the URL without changing authentication. | The tenant secret remains in the key query parameter. |
| Prompt size cap | Oversized bodies fail with 413 before upstream provider calls. | Operators can set max_prompt_bytes in config. |
| Conflict checks | Conflicting query/body model values return 400. | The proxy rejects ambiguous request shapes early. |
Use-case examples
Document summarization
A backend sends a long document as JSON prompt text instead of URL encoding it.
Generated transcript
A workflow posts messages[] with explicit order values through /v2.
Controlled token cap
A caller includes max_tokens to limit one generation.
Objections and limitations
- POST /v2 does not accept prompt or body system_prompt.
- JSON bodies still need the tenant secret in the query string.
- The proxy validates request size, but callers must still handle provider errors and timeouts.
FAQ
What is the main job of large prompt JSON POST?
LLM Proxy supports compatibility POST / with JSON prompt or messages input and canonical POST /v2 for messages-only chat transcripts.
Who should read this api contract resource?
Developers moving from quick prompt calls to large documents or generated prompt bodies.
Does this page claim provider performance or pricing advantages?
No. The supported claim is about LLM Proxy's documented routing, configuration, management, security, usage, and deployment contracts. Provider cost, speed, rankings, and benchmark claims are not made here.
Where should setup details come from?
Use the main README and implementation notes for current command, config, and endpoint details. This page summarizes the workflow without replacing LLM Proxy documentation.
What should I read next?
A closely related resource is Canonical /v2 chat messages API for LLM calls, which covers v2 chat messages API.
Related resources
Use this pattern in LLM Proxy
Start from the canonical API reference, then use the management surface when the workflow needs tenant or provider configuration.
Open API reference