Azure OpenAI: When PTU Beats Pay-as-You-Go (and When It Doesn't)
Provisioned Throughput Units guarantee latency. Pay-as-you-go is flexible and cheap. Picking the wrong one wastes either money or SLAs.
Azure OpenAI offers two purchase models. They look similar on the portal blade. They are economically and operationally very different.
Pay-as-you-go (PAYG)
- Per-token pricing.
- No throughput guarantee. You share capacity with everyone.
- Latency varies with regional load. Burst traffic = throttling (HTTP 429).
- Cheapest below ~30% utilization.
Provisioned Throughput Units (PTU)
- Reserve dedicated throughput in PTU units. Each model has a min PTU buy-in.
- Latency is bounded and predictable.
- You pay for the reservation 24/7 whether you use it or not.
- Cheaper than PAYG above ~50-60% utilization.
The break-even rule
If your sustained throughput exceeds 50% of a PTU reservation for >12 hours/day, PTU saves money. Below that, PAYG wins.
The hybrid pattern (most enterprise deployments)
Run PTU for the steady baseline of production traffic, with PAYG as overflow. Use Azure API Management as the gateway:
- Route 100% to PTU.
- If PTU returns 429 (capacity exceeded), retry to PAYG endpoint.
- Log the spillover so you size up next month.
This gives you predictable latency for normal load, infinite headroom during spikes, and bills you don't have to apologize for.
Don't forget the regional reality
PTU availability varies by region and by model. gpt-4o PTU is
often constrained in EU regions; check before you commit. Reservations
are also non-transferable across regions.