What aggregate output tokens per second means
Convert an output rate into workload demand, then check bursts, input processing, concurrency, and latency separately.
Aggregate output tokens per second measures generated tokens across requests together. A service producing 100 output tokens/s might share that output among several responses. The number alone does not tell you one response’s streaming speed, time to first token, or how many users can use the product at once.
Convert tokens/s into answers per hour
With an average answer length of T output tokens and aggregate output rate R:
answers per hour = 3,600 × R ÷ T
required tokens/s = answers per hour × T ÷ 3,600
For example, 600 answers an hour averaging 500 output tokens require 300,000 output tokens an hour, or about 83.3 tokens/s. This is an estimate of output demand. It does not test whether the service can process those prompts or finish each answer within your latency target.
Using the listed 135 tokens/s GLM‑5.3 allocation as an example:
| Average answer | Answers per hour at full output |
|---|---|
| 200 output tokens | 2,430 |
| 400 output tokens | 1,215 |
| 800 output tokens | 607.5 |
These are calculated averages, not measured completion rates. Doubling answer length halves the number of answers the same output rate represents. Continuous full output for 30 days would produce 349.92M output tokens; a monthly forecast must account for periods of lower demand and any other constraints.
An hourly average hides bursts
The 600-answer example averages about 83.3 tokens/s only if viewed across the full hour. If all 600 answers must be generated within ten minutes, the same output volume requires 500 tokens/s during that interval. The monthly token total is unchanged, but the rate needed to deliver it on time is six times higher.
Inspect the time windows that matter to the product. For interactive requests, record arrivals, overlapping requests, time to first token, and completion latency. For a batch job, record its deadline and whether queueing is acceptable. Use longer intervals to estimate cost and shorter ones to test service requirements.
Average answer length is another simplification. A mix of short answers and very long generations may have the same mean as a uniform workload while producing different queues and response times. Retain the distribution when you validate the estimate.
Output is only part of inference
Input work includes prompts, conversation history, documents, and tool results. Two requests that each generate 400 output tokens can require very different input processing. Cached input may reduce that work under the provider’s rules, but an output-only allocation does not specify those rules.
Before using an output estimate to buy capacity, confirm input throughput, context limits, concurrent requests, the measurement window, burst handling, and latency terms. User count requires additional assumptions about activity and overlap; dividing throughput by average answer length does not provide it.
Read each provider’s definition
A conversion from seconds to minutes is straightforward: multiply by 60. Comparing different providers’ capacity units requires more care. Azure PTUs use model-specific throughput and input/output weighting; Bedrock Model Units specify input and output tokens per minute across requests. Equal numeric rates can therefore represent different workloads.
Standard Thinking’s Product Plan lists aggregate output allocations. These are planning figures subject to the order’s model, allocation, and measurement conditions; input throughput, concurrency, burst duration, and latency targets are not publicly specified. Prices are confirmed before paid use.
The Product Plan capacity calculator converts an ordinary hour and a peak hour into estimated output units, then prices the resulting traffic. Bring shorter burst measurements alongside that estimate when checking whether a plan fits your product.
Sources and verification
Sources used for this resource. Verification dates describe our source checks, not provider effective dates.
Standard Thinking Product Plan
Listed model output allocations, monthly unit pricing, and extra-capacity billing support the planning examples. Prices, allocation conditions, and service commitments are confirmed before paid use.
Microsoft Azure AI Foundry provisioned throughput documentation
Provisioned deployments hold model processing capacity and bill allocated PTUs hourly. Azure Reservations discount that meter without securing deployment capacity; throughput and minimum size vary by model.
Amazon Bedrock Provisioned Throughput user guide
Model Units specify input and output throughput across requests, with hourly billing and model-dependent commitments. The documentation directs buyers to an account manager for unit specifications.
Continue exploring
$2,700a month for the example product
- Pay per token
- $4,500
- Base
- 2 GLM‑5.3 units
- Auto scaling
- 60 blocks
Calculation example, not a measurement
Estimate Product Plan output units and cost
Use traffic, answer length, and cached input to estimate monthly output capacity and compare plan cost with token charges.
34%utilization where API cost equals one unit
- Model
- GLM‑5.3
- Unit tokens/s
- 135
- API total
- $3,499.20
Calculation example, not a measurement
How to buy inference for an AI product
A buying guide to inference billing models, with a worked break-even example and the service terms to check before committing.
19.6Mtokens a month
- Input
- 18.8M
- Output
- 850K
- Model calls
- 5K
Simple tool loop · 1,000 runs a month
Estimate your monthly API cost
Estimate API usage from independent calls, growing chat history, or agent workflows, with editable rates and context assumptions.