Financial institutions have run model risk management for decades. A fine-tuned language model is a model in that sense, and validators will ask the same questions they ask of a credit model: what does it do, what data built it, how was it tested, how is it monitored, who approved it. Post-training done well produces those answers as a by-product.
Mapping SR 11-7 onto post-training#
The Federal Reserve's SR 11-7 guidance (with the OCC's parallel bulletin) frames model risk management around three activities: sound model development, implementation and use; effective validation; and governance, policies and controls. Translated to post-training:
| SR 11-7 expectation | Post-training artefact |
|---|---|
| Documented development, data and assumptions | Versioned datasets with lineage, training config, base-model licence, recipe |
| Testing and outcomes analysis | Frozen eval sets with slices and intervals; benchmarking against the incumbent process |
| Effective challenge / independent validation | Reproducible runs a validator can re-execute; held-out validation data they control |
| Ongoing monitoring | Production sampling scored by the same evaluators; drift alerts |
| Governance and approval | Release gates with named approvers; model inventory entry |
This mapping is our interpretation for practitioners, not regulatory advice; your model risk team sets the actual standard.
Where it works#
- Claims intake and extraction — dates, amounts, perils and parties from forms, emails and adjuster notes into structured records. SFT on labelled claims; GRPO with a schema + field-match reward when accuracy plateaus.
- Policy and contract clause extraction — exclusions, limits, deductibles. A deterministic field-level F1 makes validation straightforward.
- KYC and adverse-media review — summarise and classify, with a human decision. Evals focus on recall of risk indicators.
- Complaint classification and routing — high volume, clear labels, measurable.
Deployment constraints#
Most institutions require training and inference inside their own cloud accounts or data centres. POFE's VPC and on-prem deployments keep data, checkpoints and logs in your environment; the lineage graph exports as a model card for the inventory. Explainability expectations are better met with extraction and classification tasks, which have checkable outputs, than with open-ended generation.
Frequently asked questions#
Does SR 11-7 apply to LLMs?
SR 11-7 defines models broadly, and many institutions treat generative AI systems used in business decisions as models within their model risk framework. Check with your model risk management function.
Can banks fine-tune LLMs on customer data?
Typically only within their own controlled environment, with data minimisation, de-identification where possible, and documentation of data lineage. VPC or on-prem training is the usual pattern.
References#
- Board of Governors of the Federal Reserve System (2011). SR 11-7: Guidance on Model Risk Management. www.federalreserve.gov/supervisionreg/srletters/sr1107.htm