If you've decided to fine-tune open models, the next decision is how. There are three broad options — hosted fine-tuning APIs, managed post-training platforms and a do-it-yourself stack — and the right choice depends less on features than on control, data constraints and how many models you'll run. This buyer's guide lays out the trade-offs and gives you an evaluation checklist to use with any vendor, including us.
The three options#
1. Hosted fine-tuning APIs
Upload a JSONL file, choose a base model, receive a fine-tuned endpoint. Offered by model providers and inference clouds. The fastest way to a first result.
Strengths: almost no setup; inference included; good for SFT on supported models. Limits: a fixed menu of methods and hyperparameters; little visibility into training; evals mostly yours to build; data leaves your environment; weights often can't be downloaded, which ties inference to the provider.
2. Managed post-training platforms
A full workspace covering data, training, evaluation, compute and deployment, run as a service or inside your cloud. This is the category POFE belongs to.
Strengths: the whole loop in one system with lineage; all major methods including RL; bring-your-own-cloud and on-prem options; your weights. Limits: more to learn than a single API; a platform subscription on top of compute.
3. Do it yourself
Open-source libraries (TRL, Axolotl, torchtune, verl and others) on GPUs you rent or own, with your own eval, tracking and serving stack.
Strengths: total control; no platform fees; the best option for research on new methods. Limits: you build and maintain data pipelines, distributed training, checkpointing, evals, GPU procurement and serving; most of the cost is engineering time; knowledge leaves when engineers do.
Side-by-side#
| Hosted API | Managed platform | DIY | |
|---|---|---|---|
| Time to first tuned model | Hours | Hours to days | Weeks |
| Methods | Mostly SFT, some DPO | SFT, DPO family, RL (GRPO/PPO), distillation | Anything you implement |
| Model choice | Provider's catalogue | Broad open-model catalogue + BYO | Any |
| Data location | Provider | Your choice: SaaS, VPC, on-prem | Yours |
| Weights portable | Often not | Yes | Yes |
| Evaluation | Basic | Built-in, calibrated, gated | Build it yourself |
| GPU sourcing | Included, opaque | Brokered or BYO | Your problem |
| Main cost | Per-token training + inference | Subscription + compute | Engineering time + compute |
Questions that decide it#
- Can your data leave your environment? If not, hosted APIs are usually out, and you need a platform with VPC or on-prem deployment, or DIY.
- Do you need methods beyond SFT? If the task is reasoning-heavy, agentic or checkable by code, you'll want RL with custom rewards. Check that it's supported in practice, not only on a roadmap.
- How many models will you run? One model for one use case is a project; ten is a practice. Platforms pay off as the number grows, because lineage, evals and serving get reused.
- Who owns the weights? If you can't export them, your inference costs and roadmap are tied to one provider.
- Where will it be served? Make sure the training output works with your serving stack: vLLM, SGLang, Databricks Model Serving, SageMaker, Vertex AI or Kubernetes.
- What does your team want to own? A research team may prefer DIY with a platform for the boring parts; a product team usually wants the outcome.
Vendor evaluation checklist#
Use this list in any proof of concept. Ask to see each item working, on your data.
Data
- Connects to where your data lives (object storage, warehouse, Unity Catalog) without bulk copies
- PII detection and redaction with reviewable results
- De-duplication, filtering and versioned datasets
- Synthetic data and preference-pair generation
Training
- SFT, DPO-family and RL methods; LoRA and full fine-tuning; multi-node
- Custom reward functions and verifiers
- Visibility into rollouts and samples during training
- Bring-your-own training code or containers
Evaluation
- Versioned golden sets; slice-level reporting with confidence intervals
- LLM-judge calibration against human labels
- Contamination checks between training and eval data
- Release gates connected to deployment
Compute, deployment and governance
- GPU sourcing across providers, or use of your own capacity without markup
- Spot/preemptible support with automatic resume
- OpenAI-compatible serving, multi-LoRA, canary and shadow traffic
- Weights export in standard formats
- SSO, RBAC, audit logs, data residency, VPC/on-prem options
- End-to-end lineage from source data to endpoint
Running a fair proof of concept#
Most POCs measure the wrong thing — usually how quickly a demo was assembled. A fair POC fixes the outcome and measures the path:
- Choose one real use case with an existing baseline (a prompt-engineered API, a rules system or a human process).
- Build the eval first and freeze it. Share it with every vendor.
- Give each option the same data and a fixed budget of time and compute.
- Compare: eval score with confidence intervals, cost per 1,000 requests, latency, effort required from your team, and what you'd own at the end.
Two to four weeks is enough. If a vendor can't move your eval in that time with your data, that's the answer.
Total cost of ownership#
Sticker prices are hard to compare across the three options because each puts cost in a different place. Model the total cost over twelve months for a realistic scenario — for example, three production models, each retrained monthly, serving a combined few billion tokens a month.
| Cost line | Hosted API | Managed platform | DIY |
|---|---|---|---|
| Training | Per-token price × tokens × iterations | GPU at cost + subscription | GPU at cost |
| Inference | Provider's per-token price, often fixed to them | Your choice of runtime and hardware | Your runtime and hardware |
| Engineering | Low for training; evals and data still yours | Moderate: use cases, data, evals | High: platform plus use cases |
| Maintenance | None | Vendor | Your team, indefinitely |
| Switching cost | High if weights aren't exportable | Low with exportable weights | Low, but tied to your staff |
A pattern we see often: DIY looks cheapest in a spreadsheet because engineering time is treated as free. In practice, a minimal in-house stack — data pipelines, distributed training, checkpointing, evaluation, GPU procurement and serving — takes two to four experienced engineers several months to build and at least one to maintain. That cost is usually larger than a platform subscription by the second quarter.
Hosted APIs look cheapest for the first model and most expensive at scale, mostly because of inference. A fine-tuned model that can only be served by the provider that trained it inherits that provider's per-token pricing for its whole life.
Whatever you choose, make the comparison on the same scenario and include inference. It's the line that compounds.
Where POFE fits#
POFE is a managed post-training platform designed for teams that want the whole loop — data, every major training method, calibrated evals, GPU sourcing and serving — with the option to run it in their own cloud, on-prem or in Databricks, and to keep their weights. For organisations that want the outcome without building the capability first, our managed engagements deliver a production model in about six weeks.
Try the workflow in the interactive demo, compare plans and pricing, or ask for a POC on your data.
Frequently asked questions#
What is a post-training platform?
A post-training platform is software that covers the full workflow for improving pre-trained models: data curation, fine-tuning methods such as SFT, DPO and reinforcement learning, evaluation, compute management and deployment, with lineage linking them together.
Should we build our own fine-tuning stack?
Build if you have a research team, need methods no platform supports, or have unusual infrastructure constraints. Otherwise, engineering time on pipelines, evals, GPU management and serving usually outweighs platform fees within a few months.
What should a fine-tuning POC measure?
A frozen eval on your real use case, cost per request, latency, the effort required from your team, and what you own at the end — weights, data and pipelines — compared against your current baseline.
Can we switch platforms later?
Yes, if you can export weights in standard formats and keep your datasets and eval sets. Make weight portability and data ownership a hard requirement when choosing.