GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
rohanjain2312
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
rohanjain2312
Model Tree
Qwen/Qwen2.5-0.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
A three-part demo of reward hacking in GRPO:
| base model | Qwen/Qwen2.5-0.5B-Instruct |
| reward model | lvwerra/distilbert-imdb (reward = P(positive)) |
| task | continue the opening sentence of a negative IMDB review |
| algorithm | GRPO, group size 8, 8 prompts/step |
| KL coefficient (beta) | 0.0 |
| steps actually completed | 80 |
| learning rate | 1e-05 |
| max new tokens | 48 |
| metric | step 0 | step 80 |
|---|---|---|
| sentiment reward (P(positive), held-out) | 0.556 | 0.996 |
| perplexity under frozen base model | 4.5 | 18.1 |
| distinct-2 (bigram diversity) | 0.997 | 1.000 |
Full metrics and generated samples for every checkpoint are in logs/metrics.jsonl and
logs/samples.jsonl in this repo. Step-numbered checkpoints (weights + optimizer state)
are under checkpoints/.
Both runs were configured for 100 steps and stopped at step 80, when the Hugging Face Jobs credit balance on the training account ran out mid-run. Step 80 is the last completed evaluation checkpoint and is what is published here. The curves had long since separated by then (the reward saturated around step 30), so the conclusion is unaffected -- but the runs are reported as 80 steps, not 100, because that is what was run.
Research demonstration only. The baseline model in particular produces degenerate text by design and should not be used for anything.