How to buy inference for an AI product

A buying guide to inference billing models, with a worked break-even example and the service terms to check before committing.

By Standard Thinking · Updated

Pay per token is usually the simplest starting point when an AI product’s traffic is uncertain. Buying capacity becomes worth evaluating when you can describe the workload you need to serve, the hours it needs that capacity, and the cost of leaving some of it unused. The decision involves response times, peak handling, and contract terms as well as the token bill.

Match the purchase to the workload

Purchase You pay for Useful starting point Main sizing question
Pay per token Tokens consumed Variable or early traffic Monthly token volume
Dedicated deployment Running GPU instances Custom serving requirements Measured hardware performance
Provisioned throughput Allocated processing capacity Predictable production demand Provider-specific workload rate
Flat-rate subscription Plan access and limits Permitted subscription use Traffic eligibility and concurrency
Monthly throughput plan A monthly base allocation Repeatable product traffic Base rate and peaks

These are purchasing categories, not interchangeable service promises. A subscription intended for personal chat may exclude production API traffic. A GPU price buys hardware time; it tells you nothing about answers per hour until you measure the chosen model, serving configuration, prompt lengths, and concurrency. Provisioned units also differ in how they count input, output, and cached tokens.

The provider comparison puts billing meters and commitments side by side. Use it to identify the contract to investigate, rather than treating the lowest visible price as the cheapest way to serve your workload.

Separate the baseline from the peaks

Start with a traffic trace. Record requests and tokens over time, including retries and agent calls, then choose a baseline that covers the hours you intend to serve with reserved capacity. An hourly average is a useful first estimate, but shorter bursts can still create queues. Keep those bursts visible when discussing concurrency and latency requirements.

For traffic above the baseline, find out whether the provider offers a paid fallback, additional capacity, queueing, or rejection. Check whether that option has its own rate limits and whether capacity is actually available. A brief event can be expensive if it requires whole billing blocks or a large increase in units; its duration alone does not settle the cost.

Capacity that remains allocated overnight is still a cost. Moving flexible batch jobs into quiet hours may improve its use, provided those jobs fit the same model and service conditions. Do not assume that every spare hour can be filled.

Calculate break-even with your own request shape

Compare the capacity bill with the token bill for exactly the same traffic. Include ordinary input, cache reads, output, and any separately billed operations. Caching can reduce the API bill enough to change the choice, so use a measured, token-weighted cached share where possible.

For one unit, a useful reference is:

break-even utilization = monthly unit price
                       ÷ token cost at full unit output for the month

The denominator needs an input-to-output ratio and cache assumption. It is a theoretical output-based calculation, not a benchmark or proof that the workload fits the service.

For a Standard Thinking example, one listed GLM‑5.3 unit produces 135 aggregate output tokens/s. Across 30 days that corresponds to 349.92M output tokens at continuous full output. Assume four input tokens per output token, no cache reads, and our October 1, 2026 API snapshot of $1.40 input and $4.40 output per million tokens. The equivalent token bill is $3,499.20; dividing the indicative $1,200 unit price by that bill gives about 34% break-even utilization. This is an illustrative capacity calculation, not measured per-request performance.

That percentage belongs to this example. Different prices, request lengths, caches, and unit definitions produce different thresholds. With peaks, compare the complete monthly totals: base utilization alone omits the cost of extra capacity. The capacity calculator applies those inputs to one model at a time.

Check what the contract reserves

Azure illustrates an important distinction: its provisioned deployment holds processing capacity, while an Azure Reservation provides a billing discount. The financial reservation by itself does not secure deployment capacity. Other vendors’ products have their own allocation and purchase rules; the Azure wording is not a general definition of all reserved inference.

For any offer, confirm the model and version, region, measurement window, input handling, concurrency, burst behavior, latency targets, and exit terms. Also establish whether a model change requires a new commitment or sizing exercise.

Standard Thinking’s Product Plan lists a monthly output allocation with no monthly token cap. Its free extra capacity depends on shared-pool availability and is not reserved; authorized auto scaling is also subject to availability. The listed output rate does not establish input throughput or concurrent-request capacity. Indicative prices and service conditions are confirmed before paid use. Request Product Plan access with a traffic trace and your response-time requirements.

Sources and verification

Sources used for this resource. Verification dates describe our source checks, not provider effective dates.

Azure AI Foundry provisioned throughput documentation

Provisioned deployments hold model processing capacity and bill allocated PTUs hourly. Azure Reservations discount that meter without securing deployment capacity; throughput and minimum size vary by model.

Checked

Amazon Bedrock Provisioned Throughput user guide

Model Units specify input and output throughput across requests, with hourly billing and model-dependent commitments. The documentation directs buyers to an account manager for unit specifications.

Checked

Together AI pricing

The September 29 snapshot lists H100 and B200 GPU-hour prices and model-dependent PTU token types. PTU model rows were unreadable, so no PTU price is quoted.

Checked

Anthropic API service tiers documentation

The September snapshot identifies Priority Tier capacity commitments as no longer purchasable. Its model-specific input/output commitments and standard-tier fallback are historical context.

Checked

Featherless AI pricing

Chat plan $25 a month, 32K context, “4 concurrent units”, excluding “app or API traffic, reselling, background automation and benchmarking”; Business plan a custom contract “Sized to your GPUs”.

Checked

Standard Thinking Product Plan

Listed model output allocations, monthly unit pricing, and extra-capacity billing support the planning examples. Prices, allocation conditions, and service commitments are confirmed before paid use.

Checked

Standard Thinking pricing

The October 1, 2026 snapshot supplies Standard API input, output, and applicable cache-read rates for the worked calculations. These examples do not establish model availability or measured performance.

Checked

View all resources