POFE — Research, guides and changelog https://pofe.ai/feed.xml 2026-10-24T00:00:00Z POFE Research GRPO with custom Python verifiershttps://pofe.ai/changelog/#2026-10-242026-10-24T00:00:00ZReward functions can now call sandboxed verifiers (SQL execution, unit tests, schema validation) with per-call timeouts. The run view plots the share of zero-variance groups next to mean reward, and dynamic sampling can skip them. GRPO group size under a fixed rollout budgethttps://pofe.ai/research/grpo-group-size/2026-10-21T00:00:00ZA reproducible simulation of GRPO with binary verifiable rewards: how often groups carry no gradient, why smaller groups can win under a fixed budget, and what std normalisation really does. What DPO's β actually controls — and why chosen likelihoods fallhttps://pofe.ai/research/dpo-beta-and-likelihood-displacement/2026-10-14T00:00:00ZA derivation of the DPO loss and a reproducible simulation sweeping β from 0.01 to 1: reward vs KL, held-out preference accuracy, and why the log-probability of chosen responses goes down. Paired confidence intervals in eval comparehttps://pofe.ai/changelog/#2026-10-102026-10-10T00:00:00ZEval comparisons now report paired differences with 95% intervals per slice, and release gates can be written against the interval's lower bound. The release-gate checklist for fine-tuned modelshttps://pofe.ai/learn/llm-eval-release-checklist/2026-10-09T00:00:00ZA copyable checklist and YAML template for releasing a fine-tuned LLM: eval set hygiene, statistics, judge calibration, safety, operations and rollout. Built from the failures we see most. How many eval examples do you need to trust a 3-point gain?https://pofe.ai/research/how-many-eval-examples/2026-10-07T00:00:00ZPower analysis for LLM evals with a reproducible simulation: minimum detectable effect by eval size, why paired comparisons need fewer items, and a sizing table you can use before you build the golden set. H100 vs H200 vs B200 for fine-tuninghttps://pofe.ai/learn/h100-vs-h200-vs-b200-for-fine-tuning/2026-10-02T00:00:00ZWhich NVIDIA GPU to fine-tune on: published memory, bandwidth and BF16 compute for H100, H200 and B200, what each means for LoRA and full fine-tuning, and how to compare cost per run instead of cost per hour. LLM-as-a-judge agreement: what Cohen's κ hideshttps://pofe.ai/research/llm-judge-agreement-kappa/2026-09-30T00:00:00ZWhy the same LLM judge can score κ = 0.80 on one eval and 0.43 on another, how judge error biases pass rates and shrinks model gaps, and how to correct for it. Closed-form results plus simulation. Judge calibration reportshttps://pofe.ai/changelog/#2026-09-262026-09-26T00:00:00ZCalibrating a judge against human labels now reports sensitivity, specificity and κ with bootstrap intervals, and pass rates can be shown bias-corrected. When does a fine-tuned model beat the API on cost?https://pofe.ai/research/fine-tuning-vs-api-break-even/2026-09-23T00:00:00ZA transparent cost model comparing a self-hosted fine-tuned model with a hosted frontier API: the break-even token volume, the full cost curve, and which assumptions actually move the answer. How much GPU memory does fine-tuning need?https://pofe.ai/research/gpu-memory-for-fine-tuning/2026-09-16T00:00:00ZThe memory formula for full fine-tuning, LoRA and QLoRA — weights, gradients, Adam state and activations — with a computed table for 8B to 70B models and the minimum H100 count for each. Choosing a post-training platform: a buyer's guidehttps://pofe.ai/guides/post-training-platform-buyers-guide/2026-09-15T00:00:00ZHosted fine-tuning APIs, managed platforms and DIY stacks compared on control, cost, security and time-to-production. How to fine-tune an LLM on Databrickshttps://pofe.ai/guides/fine-tune-llm-on-databricks/2026-09-15T00:00:00ZUnity Catalog data, governed GPU clusters, MLflow lineage and Model Serving — the end-to-end pattern for post-training inside your lakehouse. Evaluating fine-tuned models: how to test your evalshttps://pofe.ai/guides/evaluating-fine-tuned-models/2026-09-15T00:00:00ZGolden sets, LLM-as-judge calibration, contamination checks and the regression gates that keep a release honest. How much does it cost to fine-tune an LLM?https://pofe.ai/guides/llm-fine-tuning-cost/2026-09-15T00:00:00ZA line-item breakdown of GPU hours, data, evals and serving, with worked examples for 8B, 32B and 70B models. LoRA vs full fine-tuning: quality, cost and serving trade-offshttps://pofe.ai/guides/lora-vs-full-fine-tuning/2026-09-15T00:00:00ZRank, target modules, memory math and where adapters stop being enough — with real numbers from 8B to 70B. SFT vs DPO vs GRPO: choosing a post-training methodhttps://pofe.ai/guides/sft-vs-dpo-vs-grpo/2026-09-15T00:00:00ZWhen supervised fine-tuning is enough, when preference optimization pays off, and when reinforcement learning with verifiable rewards is worth the compute. B200 capacity in the brokerhttps://pofe.ai/changelog/#2026-09-122026-09-12T00:00:00ZB200 quotes appear alongside H100 and H200, and the broker can rank by estimated cost per run instead of price per hour. Fine-tuning vs prompt engineering — and how much data fine-tuning needshttps://pofe.ai/learn/fine-tuning-vs-prompt-engineering/2026-09-12T00:00:00ZWhen a better prompt is enough, when fine-tuning pays off, and how many training examples you need for format, domain and reasoning changes. With a worked token-cost comparison. Fine-tuning vs RAG: a decision guide with failure modeshttps://pofe.ai/learn/fine-tuning-vs-rag/2026-09-09T00:00:00ZFine-tuning changes how a model behaves; retrieval-augmented generation changes what it can see. A practical comparison on accuracy, freshness, cost, latency and governance, with a decision table. What is post-training? A complete guide for LLM teamshttps://pofe.ai/learn/what-is-post-training/2026-09-02T00:00:00ZPost-training explained end to end: SFT, preference optimization, RL with verifiable rewards, distillation and evaluation — what each stage changes in the model, what data it needs and what it costs. Databricks: Unity Catalog lineage exporthttps://pofe.ai/changelog/#2026-08-292026-08-29T00:00:00ZRuns launched from a Databricks workspace register models in Unity Catalog with dataset versions and eval results attached.