To calculate LLM training cost from a GPU-hour price, turn the price into a price per token. Estimate the compute each training token needs, take the GPU's dense peak rate, scale it by the utilization that published runs report, and divide the hourly price by the tokens one GPU trains in an hour. Then multiply by the tokens in the run and allow for time lost to interruptions. Each step has a published basis, and each has a trap that moves the answer.
The method rests on a plain identity. Cottier and colleagues estimate a cloud training cost as the price per chip-hour times the training chip-hours [15]. DeepSeek-V3's report gives 180K H800 GPU hours per trillion training tokens and assumes a rental price of $2 per GPU hour [11]. That is 5.56 million tokens per GPU-hour, or $0.36 per million training tokens (this note's arithmetic) [11].
How to calculate LLM training cost
Five steps, each with its source:
- Compute per token. About 6N floating-point operations per training token for a dense model with N parameters, 2N in the forward pass and 4N in the backward pass [1, 3, 19].
- Peak rate. The GPU's dense peak at the precision the run uses [20, 22].
- Tokens per second. Model FLOPs utilization (MFU) times the dense peak, divided by 6N [3].
- Cost per million tokens. The price per GPU-hour divided by the millions of tokens one GPU trains in 3,600 seconds [15].
- Run cost. Training tokens divided by tokens per GPU-hour gives GPU-hours. Divide by the share of time spent on useful training to count lost time [7, 16].
Where a measured tokens-per-second figure exists for the same model, precision and scale, it replaces steps 1 to 3.
Compute per token and what 6N leaves out
Kaplan and colleagues estimated training compute at about 6N floating-point operations per token, counting non-embedding parameters, with the backward pass at about twice the forward pass [1]. The Chinchilla authors' fuller count, embeddings included, came within 10% of 6ND for models from 73 million to 6.8 billion parameters [2].
Two things fall outside the rule. Attention adds compute that grows with sequence length: PaLM's per-token count is 6N plus 12LHQT, for layers L, heads H, head size Q and sequence length T [3]. For the 405-billion-parameter Llama 3 model, that term adds 8.4% to 6N at 8,192 tokens and 134% at 131,072 tokens, counting full attention as PaLM's formula does (this note's arithmetic from [3, 7]). Mixture-of-experts models activate part of their weights for each token. DeepSeek-V3 has 671 billion parameters with 37 billion activated per token [11], so 6N takes the active count, as an approximation [11, 12].
A token is also a tokenizer's unit. Llama 3's tokenizer raised compression on English text from 3.17 to 3.94 characters per token, so the same text costs fewer tokens [7].
Dense and sparse peak FLOPS
NVIDIA's H100 page lists 1,979 teraFLOPS for BF16 and 3,958 for FP8 on the SXM part, with a footnote: "With sparsity" [20]. Sparsity needs two zeros in every block of four values, and NVIDIA's own A100 test of a sparse inference model found gains of up to 20% [23]. NVIDIA's preliminary H100 table at launch listed 1,000 TFLOPS for BF16, and 2,000 with sparsity [24], and the Blackwell datasheet states that dense is one-half of the sparse figure [22]. Halving gives a dense BF16 rate of 989.5 TFLOP/s for H100 SXM (this note's arithmetic from [20, 24]).
Papers measure utilization against dense peaks. Two A100 studies divide by 312 TFLOP/s [5, 6], and Llama 3's table pairs 430 TFLOPs per GPU with 43% MFU, which implies a peak of 1,000 (this note's arithmetic from [7]). A 40% MFU taken on 1,979 instead of 989.5 doubles tokens per second on paper and halves the cost per token, with no change in the hardware (this note's arithmetic from [20]). Price per FLOP across A100, H100 and B200 is set out in how to compare GPU generations on price per FLOP.
Model FLOPs utilization in published runs
PaLM defines MFU as observed tokens per second over the throughput a system at peak FLOPs would reach, counting only the forward and backward passes [3]. Hardware FLOPs utilization also counts recomputed activations, so it depends on the implementation [3]. MFU is a share of peak arithmetic. It differs from the utilization that rental analyses use for the share of hours a GPU is rented.
| Run | Hardware | Reported figure |
|---|---|---|
| Llama 3 405B pre-training | 8,192 to 16,384 H100 | 38% to 43% BF16 MFU [7] |
| 530B model, Korthikanti et al. | 2,240 A100 | 54.2% MFU [5] |
| BLOOM 176B | 384 A100 | 156 TFLOPs per GPU in its fastest configuration, half of 312 [10] |
| Llama 3.1 8B, TorchTitan | 8 or 128 H100, no Float8 | 33% to 42% MFU [14] |
| PaLM 540B (TPU, not GPU) | 6,144 TPU v4 | 46.2% MFU [3] |
The counting rule moves the number. MT-NLG's authors report 126 teraFLOP/s per A100 for a 60.1-second iteration [6], about 40% of 312 (this note's arithmetic). PaLM's authors count the same throughput as 30.2% MFU [3]. FP8 adds a second peak: TorchTitan reports no MFU with Float8 on, because the definition "is not well-defined" there [14]. NVIDIA's FP8 recipe for Llama 3 70B reaches 731 model TFLOP/s per H100, which is 37% of the dense FP8 peak or 74% of the dense BF16 peak (this note's arithmetic from [20, 26]).
A worked example with dated public prices
Take a 70-billion-parameter dense model trained on 15 trillion tokens on H100 GPUs. NVIDIA's Megatron Bridge tables report 1,664 tokens per second per GPU for Llama 3 70B at FP8 on 64 H100s [26]. That is 5.99 million tokens per GPU-hour (this note's arithmetic).
Four dated H100 prices:
- AWS lists p5.48xlarge, with 8 H100 GPUs, at $55.04 an hour on demand in US East (N. Virginia) in its price file of 10 November 2026, or $6.88 per GPU-hour [31, 32].
- Lambda lists H100 1-Click Clusters of 64 GPUs at $5.85 per GPU-hour and 256 GPUs at $5.54, for terms of 2 weeks to 1 year, viewed 8 November 2026 [30].
- Lambda lists its 8x HGX H100 instance at $4.59 per GPU-hour, and the page states that these instance rates take effect on 16 November 2026 [30].
| Price per GPU-hour | Per million tokens | 15T tokens | At 90% effective training time |
|---|---|---|---|
| $6.88 (AWS on demand) | $1.15 | $17.2M | $19.1M |
| $5.85 (Lambda, 64 GPUs) | $0.98 | $14.6M | $16.3M |
| $5.54 (Lambda, 256 GPUs) | $0.92 | $13.9M | $15.4M |
| $4.59 (Lambda 8x instance, from 16 November 2026) | $0.77 | $11.5M | $12.8M |
Arithmetic: this note's, from [26, 30, 31, 32]. The run takes 2.50 million GPU-hours. The last column uses Llama 3's "higher than 90%" effective training time [7]. On 256 GPUs the run lasts about 408 days, past the listed one-year term (this note's arithmetic from [26, 30]).
The formula route gives a slower answer. At BF16, 40% MFU on the dense 989.5 TFLOP/s gives 942 tokens per second and $1.72 per million at $5.85 (this note's arithmetic from [20, 30]). The FP8 measurement runs about 1.8 times faster, so the precision belongs in the estimate (this note's arithmetic from [20, 26]).
Rillor's GPU and neocloud pricing method brings each instance or term price to U.S. dollars per GPU-hour, one GPU for one hour, and keeps the price exactly as the provider states it on record beside it. How each provider's price data is read is covered in reading GPU prices from cloud pricing APIs.
What the GPU-hour price leaves out
Large runs stop. Llama 3 405B pre-training had 466 job interruptions in a 54-day snapshot, 419 of them unexpected, and kept effective training time above 90% [7]. In two large research clusters studied by Kokolis and colleagues, 1,024-GPU jobs had a mean time to failure of 7.9 hours [16]. Spot capacity adds reclaimed instances, covered in what research shows about training on spot GPUs.
Checkpoints, storage and data transfer are billed apart, as set out in GPU cloud costs beyond the hourly rate. Experiments cost compute too: Cottier and colleagues sample total development compute at 1.2 to 4 times the final run [15]. DeepSeek-V3's $5.576 million covers only its official training, excluding prior research and ablation experiments [11]. Across four models in Cottier's study, R&D staff, including equity, made up 29% to 49% of total amortized cost [15].
Comparing GPU classes by cost of work
For the GPT OSS 120B mixture-of-experts model at BF16 on 64 GPUs, B200 trained 13,568 tokens per second per GPU and H100 5,824 [26], a ratio of 2.33 (this note's arithmetic from [26]). The dense BF16 peak ratio is 2.27 (this note's arithmetic from [20, 22]).
| Price basis | H100 per million tokens | B200 per million tokens | B200 difference |
|---|---|---|---|
| Lambda 8x instances, from 16 November 2026 | $0.219 | $0.157 | 28% lower |
| Lambda 64-GPU clusters | $0.279 | $0.192 | 31% lower |
| AWS on demand, 10 November 2026 | $0.328 | $0.292 | 11% lower |
Arithmetic: this note's, from [26, 30, 31, 32]. NVIDIA ran its own recipes on its own systems, and the two rows use different micro-batch and pipeline settings [25, 26]. The same throughput gap gives a different saving at each provider, so the price ratio decides as much as the hardware does.
Read the full report
The full report adds each formula with its source, each run's hardware and counting rule, cross-checks from published GPU-hours, the arithmetic behind each table and the complete source list.