LLM capacity and plan terms defined

Definitions for inference capacity units, billing terms, workload estimates, and subscription limits, with provider-specific meanings kept explicit.

By Standard Thinking · Updated

Inference plans use several kinds of unit: tokens consumed, processing capacity, hardware time, and subscription allowance. A familiar abbreviation does not make two offers equivalent. Use these definitions to identify what is counted, then check the provider’s measurement and billing rules.

Capacity units

Provisioned throughput. Processing capacity allocated ahead of use for a specified service. The provider defines how requests consume it and how the allocation is billed; the provider comparison explains the different contracts.

Provisioned throughput unit (PTU). Azure’s unit of model processing capacity. Throughput and minimum deployment size vary by model, even though PTU quota is model-independent.

Model Unit (MU). Bedrock’s capacity unit, specifying input and output tokens processed across requests within a minute. Its specification depends on the model.

Generative AI scale unit (GSU). Vertex AI’s throughput unit for prompts and responses. Model-specific burndown rules convert token and other input/output types into consumption of the unit.

Token unit. In OpenAI Scale Tier, a purchased throughput bundle for one model snapshot. Some bundles count input and output separately; others combine them with weighting.

Tokens per minute (TPM). A token rate expressed per minute. It can describe demand, achieved throughput, a limit, or a purchased allocation; the abbreviation alone does not specify the token types or enforcement window.

Aggregate output tokens/s. Generated output across requests together, expressed per second. It does not specify one response’s streaming speed or time to first token.

Dedicated endpoint and GPU-hour. A deployment and its hardware-time billing unit. Converting GPU time into answers requires measured model and serving performance.

Allocation and billing

Reserved capacity. Capacity held under an agreement for a workload. Confirm what the agreement actually allocates, when it becomes available, and what service commitments apply.

Azure Reservation. A financial discount applied to the provisioned deployment’s billing meter. It does not itself secure deployment capacity; Azure advises creating the deployment before purchasing the reservation.

Spillover. Routing requests beyond a provisioned deployment to another service or deployment. Azure’s documented implementation uses a standard deployment and has model eligibility restrictions.

Pay per token. Charges based on billed tokens and their applicable rates, which can differ for input, cache reads, output, or reasoning. Other operations may be billed separately.

Base allocation. The capacity purchased for the plan’s base period. In Standard Thinking’s Product Plan, the order defines the model allocation, measurement conditions, and any service commitments.

Product Plan unit. A model’s base aggregate output rate, purchased for a month. It is not a monthly token allowance. Published rates and prices are indicative planning figures; terms are confirmed before paid use.

Auto scaling block. An interval used to bill additional capacity. Product Plan uses 2‑hour blocks after authorization, subject to availability.

Free extra capacity. In Product Plan, additional capacity available while the shared pool has room. It is not reserved and is excluded from the calculator’s cost assumptions.

Workload and cost calculations

Use of the base. Estimated output served within the base rate, divided by its output capacity over the period. Demand above that rate is excluded from base use.

Break-even utilization. The fraction of a unit’s theoretical full output at which equivalent token charges equal its price. The calculation needs prices, period length, input-to-output ratio, and cached share; a monthly comparison must also include extra-capacity charges.

Input-to-output ratio. Input tokens divided by output tokens over a defined workload. Long histories and tool results can raise the ratio even when the final answers remain short.

Cached input share. Cache-read tokens divided by total input tokens. Cache eligibility and charges follow the provider’s rules; shared prompt text does not by itself establish a billable cache hit.

Answers per hour. Aggregate output tokens/s multiplied by 3,600 and divided by average output tokens per answer. This estimates output demand or production at the stated rate, rather than validating complete workload capacity.

Busiest normal hour. A chosen baseline hour representative of recurring busy traffic. Also inspect shorter bursts and exceptional events when sizing a service; the throughput guide shows why hourly averaging can hide a peak.

Subscription limits and resets

Usage window. The interval over which a plan counts allowance, such as five hours or seven days. A billing period and a usage window can be different.

Rolling reset. Allowance becomes available according to elapsed time or activity under the provider’s stated rule. Check whether the window begins with first use or tracks each consumption event.

Fixed reset. Allowance refreshes at a specified time or subscription boundary. It is different from unused allowance carrying forward.

Credits and model multipliers. An allowance unit that can weight token types and models differently. Use the published formula before comparing credit totals with requests or tokens.

Concurrency limit. The number of requests permitted in flight together. It does not establish tokens/s or response latency.

Priority and peak restrictions. Rules for handling high demand, which may include queueing, throttling, changed prices, or preferential processing. Coding plan usage units compares documented examples.

Standard API is available. Coding Plan and Product Plan are available on request, with prices and conditions confirmed before paid use. Their configurations should be read alongside order terms.

Sources and verification

Sources used for this resource. Verification dates describe our source checks, not provider effective dates.

Azure AI Foundry provisioned throughput documentation

Provisioned deployments hold model processing capacity and bill allocated PTUs hourly. Azure Reservations discount that meter without securing deployment capacity; throughput and minimum size vary by model.

Checked

Amazon Bedrock Provisioned Throughput user guide

Model Units specify input and output throughput across requests, with hourly billing and model-dependent commitments. The documentation directs buyers to an account manager for unit specifications.

Checked

Vertex AI Provisioned Throughput overview

Provisioned Throughput described as a fixed-cost, fixed-term subscription in several term lengths that reserves throughput for supported generative AI models.

Checked

Vertex AI, calculate Provisioned Throughput requirements

GSUs measure prompt and response processing using model-specific burndown rules. Cached input has its own consumption weighting, and unused throughput does not carry over.

Checked

Vertex AI, models supported by Provisioned Throughput

The September snapshot lists supported models and model-specific throughput and burndown rates. Equal GSU counts do not represent equal output throughput across models.

Checked

OpenAI Scale Tier for API customers

The September 16 snapshot defines model-snapshot token bundles with a 30-day minimum. Token weighting differs by bundle; the published TPM figure is not universally an output-token allowance.

Checked

Anthropic API service tiers documentation

The September snapshot identifies Priority Tier capacity commitments as no longer purchasable. Its model-specific input/output commitments and standard-tier fallback are historical context.

Checked

Together AI pricing

The September 29 snapshot lists H100 and B200 GPU-hour prices and model-dependent PTU token types. PTU model rows were unreadable, so no PTU price is quoted.

Checked

Fireworks AI pricing

The September snapshot bills on-demand deployments per GPU second without startup charges. Its dated hardware price table publishes no token-throughput entitlement.

Checked

Z.ai GLM Coding Plan documentation, Overview

The credit formula with GLM‑5.3 at 6.9 input, 1.7 cached input and 24 output; off-peak usage at 50% of the credit rate, peak Monday to Friday 14:00-18:00 Singapore Standard Time (UTC+8); 5‑hour credits refresh 5 hours after consumption and weekly credits reset every 7 days.

Checked

Kimi Code documentation, Membership Benefits

Kimi Code’s rolling 5‑hour rate window; new members have no weekly quota, while legacy plans are unaffected and keep the 7‑day quota the page described on 2026-09-12 (refreshing every 7 days from the subscription date).

Checked

Claude Help Center, What is the Max plan?

The reference grammar for the five-hour and weekly pattern: the session-based limit resets every five hours, and the weekly limit across all models resets at a fixed time assigned to the account; the allowance is never an absolute unit, always a multiple of another plan’s usage per session.

Checked

Standard Thinking Product Plan

Listed model output allocations, monthly unit pricing, and extra-capacity billing support the planning examples. Prices, allocation conditions, and service commitments are confirmed before paid use.

Checked

Standard Thinking pricing

Coding Plan is available on request at $149 per 7-day period or $549 per 30-day period. Model selection may change; requests may be slower at peak hours, subject to order terms. No fixed throughput or public service-credit program is promised.

Checked

Standard Thinking Coding Plan

Coding Plan is available on request at $149 per 7-day period or $549 per 30-day period. Model selection may change; requests may be slower at peak hours, subject to order terms. No fixed throughput or public service-credit program is promised.

Checked

Standard Thinking documentation

Coding Plan is available on request at $149 per 7-day period or $549 per 30-day period. Model selection may change; requests may be slower at peak hours, subject to order terms. No fixed throughput or public service-credit program is promised.

Checked

View all resources