GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
abhishekai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
abhishekai
Model Tree
abhishekai/slm-500m-legal-sft
Input Modalities
Output Modalities
Supported Functionality
This model performs at parity with the SFT model it was trained from.
| win-rate vs slm-500m-legal-sft | 0.520 |
| tally | 16 wins / 12 losses / 72 ties (n=100) |
| two-sided sign test on the 28 decisive pairs | p = 0.572 |
| 95% CI on the overall win-rate | [0.423, 0.615] |
The interval straddles 0.5. PPO here did not measurably improve the model and did not measurably harm it. If you want the better-understood model, use the SFT one.
The 72 ties are the substantive result, not a footnote. Mean KL from the SFT reference across training was 0.134 — the policy barely moved, so the judge mostly cannot distinguish the two models. A numerically healthy PPO run at this scale produces nearly the same model.
Scored by Claude Sonnet on four axes out of 10 — a different model family from the Gemini judge behind the win-rate above, and one that had no hand in writing this project's training data. 300 held-out prompts, the same prompts and the same scale used for all twelve checkpoints, so this number is comparable across models in a way no win-rate here is.
| mean score | 4.51 / 10 |
| 95% CI (bootstrap over per-prompt scores) | [4.19, 4.84] |
| paired step | -0.15 against its own SFT (4.65), CI [-0.35, +0.06] - indistinguishable from the SFT. |
The full twelve-checkpoint table, with every stage-to-stage interval, is in
MODEL_INDEX.md in the project repository.
Same pipeline, same prompt pool, same Gemini 2.5 Flash judge, same PPO code across all three models. PPO's win-rate falls with scale — the mirror image of DPO's rise:
| model | params | PPO win-rate | p | DPO win-rate |
|---|---|---|---|---|
| 125M | 126M | 0.570 | 0.016 | 0.455 |
| 500M (this) | 518M | 0.520 | 0.572 | 0.480 |
| Gemma-2-2B | 2,614M | 0.330 | 0.0002 | 0.530 |
Only the two ends are significant, and they point in opposite directions. The Gemma PPO checkpoint is a real degradation and is deliberately not published.
The reward model distilled Gemini 2.5 Flash's preferences, PPO optimized against that reward model, and Gemini 2.5 Flash then judged the result. Win-rates under those conditions partly measure agreement with the judge's own taste. An independent judge would be needed to break that circularity.
PPO maximizing reward - kl_coef * KL(policy ‖ SFT), 60 rollout steps, 16 prompts
per rollout, kl_coef=0.2 on 1×H100. Realized: mean KL 0.134, final KL 0.556,
value loss 0.0005–0.035, kl_coef never left its floor, no abort, 13.3 min,
$0.23.
Three implementation details were load-bearing — without them the run is not merely worse, it is invalid:
kl_coef is only meaningful relative to reward
magnitude, and a Bradley-Terry reward model only fixes score differences —
its absolute scale is arbitrary. Scores are normalized per rollout batch.kl_coef, since
at step 1 the policy is the reference and KL ≈ 0.Trained from the same SFT backbone with a fresh scalar head and the pairwise Bradley-Terry loss, reaching 0.820 pairwise accuracy on held-out preference pairs. Published separately as slm-500m-legal-rm.
Use the chat template shipped with the tokenizer. The assistant turn ends on
<|eos|>.
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("abhishekai/slm-500m-legal-ppo")model = AutoModelForCausalLM.from_pretrained("abhishekai/slm-500m-legal-ppo")msgs = [{"role": "system", "content": "You are a precise legal and financial assistant. Answer only from the provided context."},{"role": "user", "content": "Context: In a civil negligence action the plaintiff must prove duty, breach, causation, and damages by a preponderance of the evidence.\n\nQuestion: What standard of proof applies to the plaintiff?"},]text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)out = model.generate(**tok(text, return_tensors="pt", add_special_tokens=False),max_new_tokens=128, do_sample=False)