How much does it cost to fine-tune an LLM?

"How much does it cost to fine-tune an LLM?" usually gets one of two useless answers: "a few dollars" or "it depends". The real answer is a short calculation, and GPU time is often not the largest line. This guide breaks the cost into its components, gives the formula for estimating compute, works through three realistic examples and lists the levers that cut the bill most.

The five cost components

A fine-tuning project that reaches production has five costs. Most budgets only include the first.

1. Training compute — GPU-hours for the runs, including the failed and exploratory ones.
2. Data — collecting, cleaning, labelling and synthesising training examples and preference pairs.
3. Evaluation — building test sets, human labels for judge calibration, and judge or verifier inference on every run.
4. Inference — serving the model in production, usually the largest cost over a year.
5. People and tooling — engineering time to build and maintain the pipeline.

A useful planning ratio for a first production model: training compute is 10–25% of total project cost. Data and evaluation together are often 30–50%. If your estimate is all GPU, it's missing most of the work.

Estimating training compute

Training a transformer costs roughly 6 × N × D floating-point operations, where N is the number of parameters being trained through and D is the number of training tokens (tokens per epoch × epochs). The 6 accounts for the forward pass (2) and backward pass (4).

Divide by what your GPUs actually sustain. An H100 delivers about 990 dense bf16 TFLOP/s on paper; well-tuned training reaches 35–50% model FLOP utilisation (MFU), so plan around 350–450 TFLOP/s per GPU.

# 32B model, 100M tokens × 2 epochs, full fine-tuning
flops      = 6 × 32e9 × 200e6         = 3.84e19
per_gpu    = 400e12 FLOP/s            # ~40% MFU on H100
gpu_hours  = 3.84e19 / 400e12 / 3600  ≈ 27
cost       = 27 × $2.50               ≈ $67

Method multipliers on top of that baseline:

Then add 2–5× for exploration. Nobody ships the first run: learning-rate sweeps, data ablations and method comparisons are normal and worth budgeting explicitly.

You can do this calculation interactively with the POFE cost calculator.

Three worked examples

1. Support-ticket classifier and responder on an 8B model

LoRA SFT on 40 million tokens for 2 epochs. Compute: 6 × 8e9 × 80e6 × 0.65 ≈ 2.5e18 FLOPs, or about 2 GPU-hours on an H100 — roughly $5 per run. With a sweep of eight runs, training compute is about $40. The real cost is the 3,000 reviewed examples that make up the dataset, and the 400 human-labelled eval items: easily 50–100 hours of expert time.

2. Contract-extraction model, 32B, SFT then DPO

LoRA SFT on 150M tokens (2 epochs), then DPO on 20,000 on-policy pairs. SFT ≈ 6 × 32e9 × 300e6 × 0.65 ≈ 3.7e19 → ~26 GPU-hours. DPO adds roughly the same. Per full pipeline run: about $130. Ten iterations over a project: ~$1,300 in GPUs. Pair labelling by lawyers: considerably more.

3. Text-to-SQL agent, 32B, SFT then GRPO with an execution verifier

Short SFT (~$30), then GRPO over 20,000 prompts with 8 rollouts each, 2 passes. Rollout generation dominates: around 150–250 H100-hours per run, or $400–$600. A project with a few reward-design iterations lands at $2,000–$5,000 in compute — still small compared with a year of frontier API calls at production volume.

Inference: the cost that compounds

Training is a one-off. Inference runs every day. The usual financial case for post-training is that a fine-tuned 8B–32B model matches or beats a much larger general model on the specific task, at a fraction of the per-token cost and latency.

Consider a workload of 2 billion tokens per month. At a blended frontier-API price of $5 per million tokens, that's $10,000 a month. A tuned 8B model on a dedicated GPU at high utilisation might cost $1,000–$1,500 a month for the same traffic. The training project pays for itself in weeks.

The cheapest inference strategies after fine-tuning are:

• Distillation: train a small student on a large teacher's outputs for your task.
• Multi-LoRA serving: many fine-tuned variants on one base model's GPUs.
• Quantisation: FP8 or INT4 weights, evaluated on your own task to confirm quality holds.
• Scale-to-zero for spiky or low-volume endpoints.

Seven levers that cut the bill

1. Fix the data before scaling compute. De-duplication alone often removes 10–30% of tokens with no quality loss.
2. Start small. Iterate on an 8B model; move to 32B or 70B only when the eval says the smaller model has plateaued.
3. Use LoRA by default. See LoRA vs full fine-tuning.
4. Pack sequences. Packing short examples into full-length sequences can double effective throughput.
5. Use spot capacity with checkpointing. Spot GPUs are often 30–60% cheaper; with frequent checkpoints, preemption costs minutes.
6. Stop early. Evaluate during training and stop when the task metric plateaus, not when the epoch ends.
7. Shop GPUs across providers. H100 prices vary by more than 2× between providers and regions on the same day.

Per-token training prices vs paying for GPUs

Hosted fine-tuning services usually charge per training token — for example a few dollars per million tokens for mid-sized models — while platforms and DIY setups charge for GPU time. Which is cheaper depends on model size, method and how much experimentation you do.

Per-token pricing is simple and predictable for SFT on small datasets: 50 million tokens at $3 per million is $150, with no infrastructure to manage. But the price per token is fixed regardless of how efficiently the job runs, and it typically covers only the methods and models the provider supports. Per-token prices for preference and RL methods, where they exist, are higher because the provider is pricing in rollout generation.

GPU-time pricing tracks real cost. With LoRA, sequence packing and good utilisation, the effective price per million training tokens on an 8B model can fall well below $0.50, and for large exploratory sweeps the difference compounds. You also pay for inefficiency, though — idle GPUs, failed runs and poorly packed batches all show up on the bill.

The hidden variable is inference. If a per-token training service only lets you serve the resulting model on its own infrastructure, the training price is a small part of the commitment you're making. Compare total cost over a year of production traffic, not the price of one run.

A budgeting template

Platforms like POFE compress the engineering line most: data pipelines, recipes, evals, GPU sourcing and serving already exist. For a quote on your own workload, send us the details.

Frequently asked questions