Self-hosted LLM vs API cost comes down to how busy the GPUs are. A rented or owned GPU costs the same each hour, busy or idle, while an API bills only for tokens used [26]. Self-hosting costs less once utilization passes a break-even level set by three other inputs: the hardware's hourly cost, the output tokens it sustains within a latency limit and the API price [26].
An eight-GPU B200 system served the open-weight model gpt-oss-120b at 87,444 output tokens per second within MLPerf's Server latency limit [6]. Rented at $6.69 per GPU-hour on 24 October 2026 [14], it costs $0.17 per million output tokens when fully busy. A host lists the same model at $0.15 per million input tokens and $0.60 per million output tokens [13]. Break-even falls at 16% to 28% utilization for zero to three input tokens per output token (this note's arithmetic from [6, 13, 14]).
Under MLPerf's tighter Interactive limit the system sustained 13,155 tokens per second [7], or $1.13 per million output tokens at full load, so over that range it never breaks even (this note's arithmetic from [7, 13, 14]).
An LLM break-even analysis in one formula
Work in dollars per million output tokens. MLPerf counts generated tokens [8], and Patil's three reference APIs price output at 5 to 6 times input [26]. Per serving unit:
- Self-hosted cost = H × 1,000,000 / (3,600 × Θ × U)
- API cost = output price + r × input price
- Break-even utilization U* = H × 1,000,000 / (3,600 × Θ × API cost)
H is the unit's cost per hour, rented [14, 15, 17] or owned [20, 21, 22]. Θ is the output tokens per second it sustains within the chosen latency limit [2, 3, 6, 7, 9]. U is the share of that capacity used over the hours paid for, and r is input tokens per output token. Self-hosting costs less when U exceeds U*. When U* is above 100%, it never does at that limit and those prices. Patil defines U as achieved over peak throughput, so assuming full use understates cost by exactly 1/U [26].
API prices: list, batch and cached input
On pages viewed 24 October 2026, three commercial model families sell batch processing at half the standard price [10, 11, 12]. Cache hits cost a tenth of base input on most models of one family, and 5% or 2.5% on some newer ones [11]. Faster service costs more: 1.8x standard for one family's priority tier [10], and 2x and 6x for another's fast tiers [12]. Prices also change on a schedule: one model's $2.00 input and $10.00 output prices double on 1 December 2026 [10].
Throughput within a latency limit
MLPerf sets two limits for each model: time to first token (TTFT) and time per output token (TPOT) [1]. A Server result is the peak Poisson arrival rate at which 99% of requests stay within both [1]. Tighter limits cut throughput, and cost per token rises by the same factor.
| System and model (MLPerf round) | TTFT / TPOT limit | Output tokens per second |
|---|---|---|
| 8x H100 (XE9680), Llama 2 70B (v5.0) | 2,000 ms / 200 ms | 30,904 |
| 8x H100 (XE9680), Llama 2 70B (v5.0) | 450 ms / 40 ms | 12,979 |
| 8x B200, gpt-oss-120b (v6.0) | 3,000 ms / 80 ms | 87,444 |
| 8x B200, gpt-oss-120b (v6.0) | 2,000 ms / 15 ms | 13,155 |
The tighter limits kept 42% and 15% of the looser figures on submitter-tuned systems [2, 3, 6, 7]. vLLM's benchmark tool measures goodput, the request rate served within TTFT, TPOT and end-to-end limits, on your own prompts [9, 24]. Memory sizing is covered in how much GPU memory LLM inference needs.
GPU utilization and LLM cost
Patil's June 2026 preprint measured one H100 serving Llama 3.1 8B, priced at $6.98 an hour, half a two-GPU node [26]. At peak throughput, cost was $0.311 per million output tokens. At one request per second it was $7.60, 24.4 times higher. Peak throughput came only where the 99th-percentile time to first token passed 5 seconds. Held to 300 ms TTFT and 50 ms TPOT at the 99th percentile, cost went no lower than $0.368 [26]. Across six configurations and all loads, cost ran from $0.21 to $15.25 [26].
In a production study of over 10 million LLM requests a day, latency-sensitive requests showed strong daily and weekly cycles, so capacity sized for the peak sits partly idle off-peak [23]. Pooling deferrable work onto the same GPUs saved up to 25% of GPU-hours [23].
Self-hosted LLM vs API cost: a worked example
The table prices eight-GPU systems at one GPU cloud's rates on 24 October 2026, $3.99 per H100-hour and $6.69 per B200-hour [14].
| System and limit | Cost per hour | $ per million output tokens, U = 100% | U = 30% |
|---|---|---|---|
| 8x H100 (XE9680), Llama 2 70B, 2,000/200 ms | $31.92 | $0.29 | $0.96 |
| 8x H100 (XE9680), Llama 2 70B, 450/40 ms | $31.92 | $0.68 | $2.28 |
| 8x B200, gpt-oss-120b, 3,000/80 ms | $53.52 | $0.17 | $0.57 |
| 8x B200, gpt-oss-120b, 2,000/15 ms | $53.52 | $1.13 | $3.77 |
This note's arithmetic from [2, 3, 6, 7, 14].
Against the same model at $0.60 per million output tokens [13], the B200 system breaks even at 28.3% utilization under the Server limit with only output charged, and at 16.2% with three input tokens per output token. Rented around the clock, at about $39,070 a month, it pays off above about 65 billion output tokens a month. Under the Interactive limit, U* is 108% to 188% (this note's arithmetic from [6, 7, 13, 14]). Patil reports that per-token providers publish no enforceable p99 latency guarantees [26], so the two sides sell different service levels.
Against a commercial model family listed at $10 per million output tokens [10, 11, 12], the system breaks even at 1.7% utilization under the Server limit (this note's arithmetic from [6, 10, 11, 12, 14]). That holds only if the open-weight model does the task as well, which no source here shows: Epoch AI found open models trailing closed ones by months, with the lag varying by domain [28, 29]. The H100 rows show the latency effect only, since this note uses no per-token price for Llama 2 70B. The rented figures leave out staff and software. Rental listings are compared in what posted GPU cloud prices tell you.
On-premise LLM cost with owned hardware
IREN reported buying 2,400 Blackwell GPUs, B200 and B300, for about $130 million including servers, network and fit-out [20], about $54,167 per GPU. Amazon cut some servers' useful life from six years to five in 2025 [21]. Take a five-year life, IREN's planning figure of 1.93 kW per GPU [20], Uptime Institute's 2025 survey average PUE of 1.54 [30] and U.S. industrial power at 8.91 cents per kWh for the 12 months to August 2026 [22]. On those inputs an owned GPU costs about $1.50 an hour. Across four to six years, PUE 1.1 to 1.54 and industrial or commercial power, the range is $1.22 to $1.96 (this note's arithmetic from [20, 21, 22, 30]).
On the B200 throughput, $1.50 an hour gives $0.038 per million output tokens under the Server limit and $0.25 under the Interactive limit, with break-even against $0.60 at 6.4% and 42.3% utilization (this note's arithmetic from [6, 7, 13, 20, 21, 22, 30]). It leaves out financing, facility space, maintenance, software and staff, which add cost, and end-of-life value, which lowers it. Rent against hardware price is set out in how rent-to-value reads GPU payback.
What two published studies assume
Pan et al. (November 2025) count only GPU purchase and electricity, and price the API at exactly the volume the hardware can produce, so the GPUs never idle in operating hours [25, 26]. Their Table IV puts break-even at 0.3 to 69.3 months, and their text's medium and large model ranges do not match it [25]. They state 160 hours a month, yet Table III capacities match 176 [25]. Patil (June 2026) measures cost against request rate, on a synthetic workload with list API prices and no discounts. It is a single-author preprint, and it reads its costs as upper bounds [26].
Prices move on both sides
At fixed capability, Epoch AI measured API prices falling 9x to 900x a year, median 50x, and cautioned that recent rapid declines may not last [27]. One GPU cloud's on-demand H100 rose from $2.95 per GPU-hour on 7 May 2026 to $4.50 on 24 October 2026, up 53% [15, 16]. The GPU cloud in the worked example lists its eight-GPU H100 and B200 rates at $4.59 and $7.69 per GPU-hour from 16 November 2026, 15% above the rates used here [14, 36]. A hyperscaler cut on-demand prices for its P5 GPU instances by up to 45% from 1 June 2025 [18]. Software moves throughput: on unchanged hardware, NVIDIA's DGX H100 served Llama 2 70B at 20,556 tokens per second in MLPerf's March 2024 round and 31,106 in April 2025 [4, 5]. A break-even point carries the date of its inputs.
Sizing the hardware, platform and models is the Design step when Rillor deploys a private superintelligence (SI) cloud: a dedicated environment on your servers, in your colocation space or on dedicated GPU capacity that Rillor sources and contracts for you.
Read the full report
The full report adds the API discount tables, rented and owned hourly costs with every input, the Pan et al. reproduction with its internal checks and Patil's load tables.