GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
CID99
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
CID99
Model Tree
mistralai/Mistral-7B-Instruct-v0.3
Input Modalities
Output Modalities
Supported Functionality
A QLoRA adapter for mistralai/Mistral-7B-Instruct-v0.3, trained with ORPO
(Odds Ratio Preference Optimization) to correct a suspected recency bias in CSRD/ESRS
sustainability classification of French regulatory filings.
The correction failed. This repository documents how it failed, with the training artifacts, evaluation predictions, and diagnostic metrics needed to reproduce the analysis.
Three signals said the training had succeeded. All three were wrong:
| Signal observed during training | What it suggested | Reality |
|---|---|---|
eval_loss decreasing normally | Healthy convergence | — |
| Preference accuracy → 100% | Objective fully learned | — |
| FNR → 0.000 across all epochs | Target bias eliminated | — |
| Actual task accuracy | — | 70.8% → 16.4% (Run 1) |
The third row is the interesting one. False Negative Rate is precisely the metric that
operationalises the hypothesis under test — and it reached its ideal value of zero. But it did
so by collapse: the model simply stopped predicting none, which drives FNR to zero
mechanically without correcting anything.
A practitioner following standard preference-tuning practice — monitor the loss, monitor preference accuracy, check the target metric — would have concluded the bias was fixed and shipped a model that had lost 54 accuracy points.
This is a concrete instance of Goodhart's law on a real regulatory task, with the data to back it.
Evaluated on 140 expert-annotated paragraphs (Gold Standard of FinCAC40), never seen during training.
| Accuracy | Macro-F1 | κ [95% CI] | Malformed output | Global FPR | |
|---|---|---|---|---|---|
| Base model (3-shot) | 70.8% | 18.7% | 0.193 [0.051, 0.352] | 0% | — |
| ORPO — Run 1 | 16.4% | 23.1% | n/a | n/a | — |
| ORPO — Run 2 (this release) | 35.7% |
κ is Cohen's kappa on the binary CSRD vs. none decision, with bootstrap confidence intervals
(B = 2000, seed 42).
Read the confidence intervals. κ moves from 0.193 to 0.238, but the intervals overlap heavily. On n = 140 this apparent gain is indistinguishable from sampling noise, and we do not claim it as an improvement.
1. Refuge category. ESRS2 (general governance) simultaneously reaches 5.3% precision
(1 correct out of 19 predictions) and 5.9% recall (1 recovered out of 17 true instances). Both
near zero at once means the label carries no discriminative signal: the model uses it as a
default when uncertain, triggered notably by the generic word "risque" appearing in
standardised legal disclaimers.
2. Capability regression. The tuned model emits unparseable JSON in 30.7% of cases, against 0% for the base model. Fine-tuning did not merely fail to correct the target bias — it degraded a formatting capability the base model had fully mastered.
Four cumulative factors, not alternatives:
mistral-large-latest judged
open-mistral-7b; a shared bias between them cannot be excluded.We attribute the failure to this constrained-data regime, not to a limitation of ORPO as a method.
| Base model | mistralai/Mistral-7B-Instruct-v0.3 |
| Method | ORPO (Hong et al., 2024) |
| Quantization | QLoRA 4-bit NF4, double quant, bfloat16 compute |
| LoRA | r = 16, α = 32, dropout 0.05, all linear projections |
| ORPO β | 0.1 |
| Preference pairs | 225 type-A / 30 type-B / 15 type-C (after rebalancing) |
| Seed | 42 |
Preference pairs were built by rejection sampling: for each paragraph, a silver label from a
stronger judge was compared against the base policy's own zero-shot prediction. Disagreements
became chosen/rejected pairs, so every pair captures a genuine model error rather than a
fabricated counterexample.
Intended. Research on preference-optimization failure modes; reproducing the collapse; studying evaluation blind spots in alignment training; teaching material on Goodhart's law.
Out of scope. Any production classification. Any regulatory, compliance, audit, or investment decision. Any use where a 30.7% malformed-output rate is not acceptable — which is essentially all of them.
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelbase_id = "mistralai/Mistral-7B-Instruct-v0.3"tok = AutoTokenizer.from_pretrained(base_id)model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")model = PeftModel.from_pretrained(model, "CID99/Mistral-7B-ORPO-CSRD")# Expect over-prediction of CSRD categories and ~30% malformed JSON.# For a working classifier, drop the PeftModel line and use the base model.
The exact system prompt used for every evaluation reported above is in prompt_template.txt
in this repository. Reproducing our numbers requires that prompt, temperature 0, and seed 42.
adapter_model.safetensors, adapter_config.json — the LoRA adapterprompt_template.txt — the system prompt used across all evaluationstraining_log.json — full training telemetry (the metrics that looked healthy)orpo_pairs.jsonl — the preference pairs used for trainingeval_predictions.csv — per-example predictions on the 140 Gold paragraphseval_report.json — all metrics reported abovebibtex
@misc{dieng2026fincac40,title = {FinCAC40 : un corpus réglementaire français (2010--2026) pour l'évaluationde la robustesse temporelle des LLM en classification de durabilité},author = {Dieng, Cheikh Ibra},year = {2026},note = {Preprint}}
| 35.9% |
| 0.238 [0.079, 0.387] |
| 30.7% |
| 64.9% |