GPU price performance over time is usually quoted as one rate: FLOP/s per dollar doubling every so many years. For data-center GPUs, that rate rests on three choices. The first is the FLOP counted, with its precision and whether it assumes sparsity. The second is spec-sheet peak or measured throughput. The third is the price, its kind and its date. This note compares A100, H100 and B200 on stated bases, from vendor datasheets, one MLPerf® Inference round [21] and dated public prices.
Every worked figure below is this note's arithmetic from the cited sources and illustrates the method. None is a Rillor Compute Index value, and the note makes no price forecast.
GPU price performance over time: what the trend figures measure
Epoch AI's estimates each answer their own question.
- 2022 report. For 470 GPUs released from 2006 to 2021, FLOP/s per dollar doubled every 2.46 years at FP32. GPUs used in ML research doubled every 2.07 years and the top GPUs every 2.95 years [1].
- 2024 data insight. Hardware "at any given precision and fixed performance level becomes 30% cheaper each year" [3].
- 2026 data insight. Since 2023, chips bought each quarter yielded about 49% more performance per dollar each year in constant 2025 dollars, a doubling every 1.7 years [4]. It counts spec-sheet peaks against card or module purchase prices, weighting each chip by its share of each quarter's spending [4].
The 2026 series was nearly flat at 6% a year from 2023 to mid-2024, then "roughly doubled as Blackwell-generation chips grew to a majority of new spending" [4]. A spending-weighted series measures the mix bought, so it moves when buyers switch generations. Epoch's 2022 sample shows the same effect: it suggests "relevant labs spending more resources on procuring top GPUs over time" as one reason ML GPUs improved faster [1].
Quote each estimate with its sample, FLOP basis, price basis and deflator. A trend spliced from two of them has no basis.
Cost per FLOP starts with which FLOP
NVIDIA's H100 page prints its TF32, BF16, FP16, FP8 and INT8 tensor rates with sparsity, and only its FP64 Tensor Core rate dense [9]. Sparse Tensor Cores handle a 2:4 pattern "where two of every four values are zero" and do the same effective work in half the time, so the dense rate is half [14]. The network has to be pruned first, and on one A100 benchmark the measured gain was up to 20% [14]. Use dense rates unless the workload is pruned.
| GPU | FP16 or BF16, dense | FP8, dense | FP4, dense |
|---|---|---|---|
| A100 SXM | 312 TFLOPS | No FP8 row | No FP4 row |
| H100 SXM | 989.5 TFLOPS | 1,979 TFLOPS | No FP4 row |
| B200 (HGX) | 2,250 TFLOPS | 4,500 TFLOPS | 9,000 TFLOPS |
Sources: [7, 9, 11, 14]. H100 and B200 are halved from the sparse figures, as the B200 datasheet and NVIDIA's sparsity guide state.
At dense FP16, H100 is 3.17 times A100, and B200 is 2.27 times H100 (this note's arithmetic) [7, 9, 11]. Set H100's sparse FP8 figure against A100's dense FP16 and the step reads 12.7 times (this note's arithmetic) [7, 9]. Each generation adds a lower precision, so counting each at its new format inflates the gain.
A lower-precision FLOP is also a different unit of work. NVIDIA reports its NVFP4 format at "1% or less accuracy degradation versus FP8" on one benchmark set for a reasoning model, and marks FP4 and MXFP4 with a "Risk of noticeable accuracy drop compared to FP8" [15]. Halving is not universal either: the B300 datasheet prints dense FP4 as 14 PFLOPS beside 18 sparse [12]. Rates by board, system and form factor are in HGX H200, B200 and B300 compared.
One fix is a precision-normalized metric. The Bureau of Industry and Security defines total processing performance (TPP) from dense rates times bit length [17], and Epoch's 2026 insight uses it [4]. On dense rates, H100 scores 15,832 at both FP8 and FP16 (this note's arithmetic) [9].
Peak or delivered: MLPerf® throughput
Peak rates ignore memory. In MLPerf® Inference v5.0, one submitter's eight-GPU systems give a direct test [22]:
- H200 has H100's tensor rates with 1.76 times the memory [9, 10]. On a 70-billion-parameter language model benchmark, eight H200s produced 1.12 times the Offline tokens per second of eight H100s [22].
- On a text-to-image benchmark at FP8, eight B200s produced 1.60 times eight H200s, against a dense FP8 peak ratio of 2.27, a delivered ratio about 70% of the peak ratio (this note's arithmetic) [10, 11, 22].
- On the 70-billion-parameter benchmark, eight B200s at FP4 weights produced 3.16 times eight H100s at FP8 [22].
Training reports count dense peaks too. A 405-billion-parameter model's pre-training on H100s reported 38% to 43% BF16 model FLOPs utilization, with 430 TFLOPs per GPU at 43% [19]. That is 43.5% of H100's dense BF16 rate and would read 21.7% against the sparse figure (this note's arithmetic) [9, 19].
MLPerf® notes: MLPerf® Inference: Datacenter v5.0, Closed division, Available category, benchmarks llama2-70b-99.9 and stable-diffusion-xl, Offline scenario. Result IDs 5.0-0056, 5.0-0057 and 5.0-0060, retrieved 8 November 2026 from the MLCommons results repository [22]. Ratios and the cost per token below are this note's arithmetic and are not MLPerf® metrics. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.
Which price: launch, estimate or hourly rent
Purchase evidence comes in kinds. DGX A100 launched in May 2020 at a starting price of $199,000 for eight GPUs [8]. Epoch uses $10,000 to $15,000 for an A100 in 2022 and 2023, and $27,000 to $35,000 for H100 in 2023 [5]. CNBC reported NVIDIA's chief executive estimating $30,000 to $40,000 for one Blackwell chip in March 2024 [24], a range Epoch applies to standalone B200s [5]. Per dense FP16 TFLOP/s, that is $32 to $48 for A100, $27 to $35 for H100 and $13 to $18 for B200 (this note's arithmetic) [5, 7, 9, 11, 24]. None is a transaction price.
Rents vary by provider and pricing type for the same GPU class. Azure's on-demand H100 rate in East US is 1.79 times AWS's in US East, and 5.4 times Azure's own spot rate (this note's arithmetic) [26, 28]. Rents also move with no change in hardware: AWS cut on-demand P5 (H100) prices by 44% from 1 June 2025 [25]. How each provider's price file is read is in reading GPU prices from cloud pricing APIs.
FLOPS per dollar across three generations, from posted rents
US dollars per dense PFLOP/s-hour, price per GPU-hour divided by dense PFLOP/s:
| Price source | A100 FP16 | H100 FP16 | H100 FP8 | B200 FP16 | B200 FP8 |
|---|---|---|---|---|---|
| AWS on-demand, US East, price file of 10 November 2026 | $8.80 | $6.95 | $3.48 | $6.33 | $3.16 |
| Lambda 8x plan, rates from 16 November 2026 (A100 40 GB) | $6.38 | $4.64 | $2.32 | $3.42 | $1.71 |
Sources: [7, 9, 11, 26, 27, 32]. The arithmetic is this note's.
At the same precision, B200 costs 9% less per dense FLOP than H100 at AWS and 26% less at Lambda (this note's arithmetic) [9, 11, 26, 27, 32]. H100 costs 21% less per dense FP16 FLOP than A100 at AWS, while at Azure's East US on-demand rates it costs 14% more (this note's arithmetic) [7, 9, 26, 27, 28, 29, 30, 31]. Counting B200 at FP4 against H100 at FP8 shows 54% less at AWS (this note's arithmetic) [9, 11, 26, 27], a gap that counts a precision change as a price change.
On delivered throughput, the 70-billion-parameter results priced at AWS on-demand rates give $0.49 per million tokens for eight H100s and $0.32 for eight B200s, 34% less, with FP4 weights against FP8 [22, 26]. This derived cost metric pairs NVIDIA's DGX systems with AWS instances of the same GPU count. The same per GPU-hour rents feed the method in rent or buy GPUs: a break-even method.
Quality change: what a matched-model series holds fixed
The U.S. Bureau of Labor Statistics separates price change from quality change [33]. A matched-model index follows the same item over time. When the item leaves the market, a less comparable replacement can bring "quality change and an associated price differential into the index" [33]. The hedonic method removes the estimated value of a quality change from the price comparison [33]. Byrne, Oliner and Sichel found their preferred index of microprocessor prices falling 43% a year from 2008 to 2013, while the producer price index fell 8% [36].
Price per FLOP is a one-characteristic quality adjustment: it treats peak FLOP/s as the only quality that counts. H200 shows the gap. Price per peak FLOP counts it as an H100, so any premium for its extra memory reads as a higher price per FLOP, yet it produced 1.12 times the tokens per second [22].
For GPU rents, a matched-model series follows one GPU class and variant at one provider and pricing type, so its changes are price changes [33, 35, 36].
The Rillor Compute Index measures what GPU compute costs to rent and to own under a published method, with RCI Rental broken out by GPU class, region, term and pricing type and RCI Hardware by comparable system grade. Which price each rental index counts is covered in GPU rental price indexes compared.
Read the full report
The full report adds every trend estimate with its sample and price basis, dense and sparse rates for five GPUs, the MLPerf® results with their IDs, purchase and rental evidence by kind, the quality-adjustment methods of the U.S. Bureau of Labor Statistics, a seven-step checklist and the complete source list.