The ROI of post-training: when a fine-tune pays back

The return on investment of post-training is the value a fine-tuned model creates (lower inference cost, fewer failures, more revenue) set against what it costs to build and keep running (data, training compute, evaluation, engineering, monitoring and retraining). A fine-tune is a capital project with an upfront cost and a stream of monthly savings, and it can be analysed like one: with a cash-flow model, a payback period, a net present value, and a sensitivity analysis that shows which assumptions the decision actually rests on. This article builds that model for a concrete case, derives the payback and optimal-dataset-size formulas, and runs a Monte Carlo over the uncertain inputs. The result is a decision rule more specific than "fine-tune at scale": under our assumptions the decision hinged on volume, the cost of a failure and whether the fine-tune actually beats the baseline, and a cheap pilot that measures the last of these was worth more than any other step.

A fine-tune is an investment, not a line item#

The other articles in this series compare options at a point in time: the quality-adjusted cost per task of a fine-tuned model against a frontier API, and the monthly TCO of serving it. Both treat the training project as a fixed cost amortised over twelve months. That hides the questions a budget owner asks: how much do we spend before we see any return, how many months until we are even, what is it worth over the life of the model, and what has to be true for it to be worth doing at all. Answering them needs the time dimension, because costs come first and savings arrive gradually as traffic moves over.

Sculley et al. (2015) are the standard warning against counting only the build: machine-learning systems accumulate maintenance cost through data dependencies, configuration and feedback loops, and that cost recurs. In the model below, recurring costs (monitoring time and periodic retraining) turn out to move the payback period more than the training compute does.

The cash-flow model#

Costs

The upfront investment for a fine-tune trained on $n$ labelled examples at $c_{\text{ex}}$ each, with training compute $C_{\text{gpu}}$ (including hyperparameter sweeps), evaluation $C_{\text{eval}}$ (building a held-out set and running it) and $E_0$ engineer-months at loaded monthly cost $w$, is

Each month after launch carries monitoring effort $m$ FTE, a serving fixed cost $F_t$, and a retraining cost $R_c$ every $T_r$ months to keep up with drift and base-model upgrades.

Benefits

Per task, the fine-tune saves the difference in inference cost $k_b-k_t$ against the baseline, and the difference in failures $(q_t-q_b)c_f$, where $q$ is the success rate and $c_f$ the cost of a failed task, exactly as in the quality-adjusted model. If quality also moves revenue (conversion, retention), add $v$ dollars per task per point of success rate. Retiring the baseline also removes its fixed cost $F_b$. With $N_t$ tasks in month $t$, ramping linearly to $N$ over $\mu$ months, the net cash flow is

Payback and NPV

The payback period is the first month at which cumulative net cash turns positive, $t^{\ast }=\min \{t:{\sum }_{\tau \le t}{B}_{\tau }\ge I_0\}$. If monthly net cash at full volume is a constant $\bar{B}$ (with retraining spread evenly) and the ramp is linear over $\mu$ months, the ramp months jointly deliver $\bar{B}(\mu -1)/2$ less than full months would, so

The net present value over a horizon of $T$ months at monthly discount rate $r$ is $\text{NPV}=-I_0+{\sum }_{t=1}^T{B}_t/(1+r{)}^t$. We use $T=24$ months, because a fine-tune rarely outlives two base-model generations without being rebuilt, and 10% a year.

How quality depends on data

The success rate is not a constant you choose; it rises with data and saturates. We model it as a saturating power law, the shape that scaling studies find for loss against data (Kaplan et al., 2020) and that Hernandez et al. (2021) found for fine-tuning on top of pretraining, where pretraining acts like a multiplier on the fine-tuning set:

$q_0$ is the prompted model before training, $q_{\max }$ the ceiling the model class can reach on this task, $n_0$ a scale constant and $\alpha$ the rate. These are assumptions to be fitted on your own learning curve: train on 250, 500, 1,000 and 2,000 examples, evaluate each, and fit the three parameters. LIMA (Zhou et al., 2023) is a reminder that the curve can be steep early when data is carefully curated: 1,000 curated prompt-response pairs were enough to teach a strong base model specific response formats.

Setting the derivative of NPV with respect to $n$ to zero gives the optimal dataset size. Ignoring the ramp, with annuity factor $A_T={\sum }_{t=1}^T(1+r{)}^{-t}$:

The left side is the present value of one more example's quality gain across all future tasks; the right side is its price. Data is worth buying until the two are equal, so higher volume and costlier failures justify larger datasets, and the optimum scales as $(N{c}_f/c_{\text{ex}}{)}^{1/(1+\alpha )}$.

Assumptions for the base case#

The base case continues the ticket-triage task from the quality-adjusted cost model: the baseline is a frontier API, the fine-tune a post-trained 8B open model served per token. All values are illustrative assumptions.

Under this curve the fine-tune reaches 0.867 at 250 examples, 0.895 at 500, 0.921 at 1,000 (level with the baseline), 0.941 at 2,000, 0.954 at 4,000 and 0.968 at 16,000.

Worked example by hand#

Upfront: data $4000\times 2.50=\$10,000$, compute $\$3,000$, evaluation $\$4,000$, engineering $1.5\times 22,000=\$33,000$: $I_0=\$50,000$. Two-thirds of it is people.

Per task: inference saving $0.0066-0.000088=\$0.0065$; quality saving $(0.954-0.920)\times 1.50=\$0.0513$; in total $\beta =\$0.0578$. The quality term is eight times the inference term. At 300,000 tasks a month that is $\$17,346$; add the baseline's $\$1,500$ fixed cost no longer paid, subtract $\$500$ serving, $0.15\times 22,000=\$3,300$ monitoring and $8,000/4=\$2,000$ average retraining: $\bar{B}=\$13,046$ a month.

The month-by-month model, with the retraining spend landing in months 4, 8 and so on, gives 5.0 months. Over 24 months cumulative net cash reaches $246,000 and the NPV at 10% a year is $217,000.

Results#

Volume decides whether it pays back at all

At 600,000 tasks a month the project pays back in 2.6 months with a 24-month NPV of $578,000. At 300,000, 5.0 months. At 150,000, 13.0 months and an NPV of $37,000, a marginal investment. At 50,000 it never pays back: the monthly benefit of $2,900 does not cover $4,300 of net recurring cost (monitoring, serving and retraining, less the baseline's fixed cost no longer paid), so cumulative cash falls further every month and the NPV is −$83,000. Below some volume a fine-tune is a running loss, not a slow payback, because its recurring costs do not shrink with traffic. In this model that volume is $4,300/0.0578\approx 74,000$ tasks a month.

How much data to buy

Below about 1,000 examples the fine-tune does not beat the baseline and is not deployed. Above that, NPV rises steeply and flattens. The optimum sits at about 45,000 examples at $0.50 each (NPV $370,000), 18,500 at $2.50 ($316,000) and 8,500 at $10 ($227,000). The closed-form condition gives 47,000, 19,000 and 8,700 for the same three prices; the small gap is the adoption ramp, which the formula ignores. Two practical readings follow. First, at realistic labelling prices the value-maximising dataset is several times larger than the minimum that beats the baseline, because each point of success rate is worth $A_{T}N{c}_f/100$, here about $98,000 in present value. Second, the NPV curve is flat near its peak: 4,000 examples at $2.50 already captures $217,000 of the $316,000 optimum, and the remaining gain is less certain than the curve suggests. Buy data in rounds, refit the learning curve after each, and stop when the measured marginal gain falls below the formula's threshold.

What the payback period rests on

Four inputs can make the project fail outright: volume (no payback at 50,000 tasks a month; 1.9 months at a million), failure cost (never at $0.30; 2.0 months at $5), baseline success rate (2.6 months if the frontier API only reaches 0.88; never if it reaches 0.95), and the quality ceiling of the fine-tune (never at 0.93; 4.2 months at 0.985). Every other input moves payback by a few months at most: monitoring effort from 4.4 to 8.7 months, upfront engineering from 3.8 to 7.4, the adoption ramp from 3.7 to 6.7, data price from 4.5 to 7.0, retraining cost from 4.7 to 6.1. The frontier API's own price, halved or doubled, moved payback only from 5.3 to 4.5 months, because the inference saving is a small part of $\beta$. Training compute was too small to include: at $3,000 it is 6% of the upfront cost.

Notice that three of the four decisive inputs (failure cost, baseline success and the fine-tune's ceiling) are measurable before the project starts, with an evaluation set and a few small training runs. Volume is usually known.

Uncertainty and the value of a pilot

We drew 20,000 scenarios (seed 2026) at each of three known volumes, sampling the uncertain inputs over their ranges: failure cost log-uniform from $0.30 to $5, quality ceiling and rate, baseline success, engineering, data price, monitoring, retraining cost and ramp. In each scenario the team deploys only if the fine-tune's steady-state monthly net cash is positive; otherwise it abandons after spending $I_0$.

At 300,000 tasks a month, 22% of scenarios paid back within 6 months, 34% within 12 and 42% within 24; in the other 58% the fine-tune was not enough better than the baseline to cover its recurring costs and was abandoned. The NPV distribution was strongly skewed: 10th percentile −$77,000, median −$36,000, 90th percentile $506,000, mean $107,000. At a million tasks a month, 71% paid back within two years, the median payback was 5.1 months and the mean NPV $715,000. At 150,000, only 23% paid back and the mean NPV was −$4,800.

A positive mean with a negative median is the signature of a bet whose downside is bounded and whose upside is large, and it is exactly the situation in which buying information first pays. Suppose an $8,000 pilot (a few hundred examples, a small training run and the evaluation set) reveals whether the full project would clear the bar, and its work is reused if the project proceeds. Then abandoned scenarios lose $8,000 instead of about $58,000. In the simulation that raised the mean NPV at 150,000 tasks a month from −$4,800 to +$34,000, at 300,000 from $107,000 to $136,000, and at a million from $715,000 to $730,000. The pilot turns a marginal project into a positive one at moderate volume and is cheap insurance at high volume.

How ROI estimates go wrong#

Counting only the token saving. In this model the inference saving was one-eighth of the per-task benefit. An ROI case built on token prices alone either understates a good project or, if the fine-tune is less accurate than the baseline, approves a bad one. Price failures.

Assuming the quality gain instead of measuring it. The Monte Carlo's main finding is that whether $q_t$ beats $q_b$ by enough decides the project. Published comparisons (Bucher and Martini, 2024; Hsieh et al., 2023) show fine-tuned small models often win on narrow tasks, but "often" is not your task. Measure it with an evaluation set large enough to detect the gap you need; see how many eval examples.

Leaving out recurring costs. Monitoring and retraining were the costs that made low-volume projects lose money every month. A business case with no line after launch is incomplete.

Ignoring the ramp. Traffic moves to a new model gradually, behind feature flags and A/B tests. Each month of ramp adds about half a month to payback.

Comparing against a moving baseline as if it were fixed. If the baseline API improves during the project, $q_b$ rises and the benefit shrinks. The tornado shows that a baseline at 0.95 instead of 0.92 erases the return. Re-run the case at each retraining point against the current baseline.

Decision guide#

Implementation sketch#

def q_of_n(n, q0=0.80, qmax=0.975, n0=300, alpha=0.8):
    return qmax - (qmax - q0) * (1 + n / n0) ** (-alpha)

def cash_flows(N=300_000, ramp=3, k_b=0.0066, q_b=0.92, F_b=1500, k_t=0.000088, F_t=500,
               c_f=1.50, n=4000, c_ex=2.50, compute=3000, eval_cost=4000, eng_months=1.5,
               fte=22_000, monitor=0.15, retrain=8000, retrain_every=4, T=24):
    beta = (k_b - k_t) + (q_of_n(n) - q_b) * c_f          # $ saved per task
    flows = [-(n * c_ex + compute + eval_cost + eng_months * fte)]
    for t in range(1, T + 1):
        N_t = N * min(1, t / ramp)
        flows.append(N_t * beta + F_b - F_t - monitor * fte - (retrain if t % retrain_every == 0 else 0))
    return flows

def payback_months(flows):
    cum = 0.0
    for t, x in enumerate(flows):
        if t and cum + x >= 0:
            return t - 1 + (-cum) / x
        cum += x
    return float("inf")

def npv(flows, annual_rate=0.10):
    r = (1 + annual_rate) ** (1 / 12) - 1
    return sum(x / (1 + r) ** t for t, x in enumerate(flows))

f = cash_flows()
print(round(payback_months(f), 1), round(npv(f)))      # about 5.0 months, about 217,000

The function is deliberately small enough to put in a planning document. The inputs that decide the answer, $q_b$, the learning curve and $c_f$, come from evaluation; in POFE a training run reports the held-out success rate next to the baseline's on the same set, which is the number this model needs most.

Further reading#

Sculley et al. (2015) is the classic account of the recurring costs of machine-learning systems. Kaplan et al. (2020) and Hernandez et al. (2021) establish the power-law behaviour behind the learning-curve model, the second specifically for fine-tuning. Zhou et al. (2023) show how far a small curated dataset can go. Hsieh et al. (2023), Bucher and Martini (2024) and Le Scao and Rush (2021) are evidence on when small trained models beat large prompted ones and what a prompt is worth in data. For the inputs to $\beta$, see cost per task and the quality-adjusted comparison; for the training methods themselves, SFT, LoRA and DPO.

Frequently asked questions#

References#

1. D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, et al. (2015). Hidden Technical Debt in Machine Learning Systems. papers.nips.cc/paper_files/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html
2. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei (2020). Scaling Laws for Neural Language Models. arxiv.org/abs/2001.08361
3. Danny Hernandez, Jared Kaplan, Tom Henighan, Sam McCandlish (2021). Scaling Laws for Transfer. arxiv.org/abs/2102.01293
4. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. (2023). LIMA: Less Is More for Alignment. arxiv.org/abs/2305.11206
5. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, Tomas Pfister (2023). Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes. arxiv.org/abs/2305.02301
6. Martin Juan José Bucher, Marco Martini (2024). Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification. arxiv.org/abs/2406.08660
7. Teven Le Scao, Alexander M. Rush (2021). How Many Data Points is a Prompt Worth?. arxiv.org/abs/2103.08493

Related reading