Usage
Normalized token usage metadata across providers
Expose request, response, and total token counts through common headers and JSON fields.
Teams that need operational usage signals without parsing every provider's response shape.
The problem
Providers report usage differently, and response format choices can make token accounting disappear from caller code.
How LLM Proxy helps
LLM Proxy normalizes upstream token usage into common response headers and JSON usage fields when providers return token metadata.
How it works
- Call GET, POST, or /v2 with the desired response format.
- Read X-LLM-Proxy-Request-Tokens, X-LLM-Proxy-Response-Tokens, and X-LLM-Proxy-Total-Tokens when present.
- Use JSON format when the caller wants usage inside the response body.
- Review management usage dashboards for managed tenant aggregates.
Feature-to-benefit table
| Feature | Why it matters here | Example |
|---|---|---|
| Common headers | Token counts stay available for plain text, CSV, and XML bodies. | The body format does not erase usage metadata. |
| JSON usage object | JSON responses include normalized request, response, and total counts. | Callers can store one shape. |
| Managed usage aggregation | Usage events record normalized token counts without storing prompts or responses. | Dashboards can show token totals for the selected all-time, 30-day, 7-day, or 1-day interval. |
Use-case examples
Plain text caller
A CLI reads the text body and token headers separately.
Dashboard metric
The management UI shows request and token graphs for the signed-in tenant.
Provider comparison
A team compares usage by provider and model using the normalized metadata available in management mode.
Objections and limitations
- Usage metadata appears only when the upstream provider returns token information.
- Normalized counts are operational signals, not pricing calculations.
- Management usage stores metadata and excludes prompts, transcripts, responses, provider keys, and tenant secrets.
FAQ
What is the main job of normalized token usage metadata?
LLM Proxy normalizes upstream token usage into common response headers and JSON usage fields when providers return token metadata.
Who should read this usage resource?
Teams that need operational usage signals without parsing every provider's response shape.
Does this page claim provider performance or pricing advantages?
No. The supported claim is about LLM Proxy's documented routing, configuration, management, security, usage, and deployment contracts. Provider cost, speed, rankings, and benchmark claims are not made here.
Where should setup details come from?
Use the main README and implementation notes for current command, config, and endpoint details. This page summarizes the workflow without replacing LLM Proxy documentation.
What should I read next?
A closely related resource is Account-wide managed tenant usage dashboard for LLMs, which covers managed tenant usage dashboard.
Related resources
Use this pattern in LLM Proxy
Start from the canonical API reference, then use the management surface when the workflow needs tenant or provider configuration.
Open API reference