GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
gyanbhambhani
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
gyanbhambhani
Model Tree
gyanbhambhani/oss-jev-qwen05-decision-noul-bin
Input Modalities
Output Modalities
Supported Functionality
Use for
Answer:\n promptingDon't use for
noul_binary_head.pt as the decision surfaceLimits
| Dataset | gyanbhambhani/oss-jev-distill |
| Scale | ~10k expanded train (negation pairs); Hub pin era a088111937a3 |
| Teacher | jev-1.13.0 silver hard codes + soft probs |
| Eval | Frozen 98 rows held out of training |
| Mix | Choice 28 · Noul 43 · Score 27 |
| Metric | Argmax vs teacher hard labels; Noul @ P(Y)≥0.5 |
| Prompt | ends with Answer:\n |
| Bias | 0 (raw LM argmax) |
| Primary surface | LM Y/N |
| Metric | Value |
|---|---|
| Overall | 75.5% (74/98) |
| Choice | 82.1% (23/28) |
| Noul | 81.4% (35/43) — 3 Y / 40 N |
| Score level | 59.3% (16/27) |
| Score Pearson r | ~0.69 level-proxy / ~0.77 continuous |
| P(Y) mean / min / max | 0.349 / 0.013 / 0.853 |
Train: continue 4000 steps from noul-bin, train_loss≈0.7113.
Job 6ad2748b5ccb47ba0f7b4f1e · a10g-small · ~35 min.
Hub sha cfd24b3ffae6bcd726bee3b73fed68743b4a8b4c.
| Condition | Overall | Choice | Noul | Noul preds | Score lvl |
|---|---|---|---|---|---|
| 10k baseline | 45.9% | 82.1% | 16.3% | 43/43 Y | 55.6% |
| noul-aug | ~46.9% | 85.7% | 16.3% | 43/43 Y | 55.6% |
| noul-bin LM Y/N | 60.2% | 82.1% | 44.2% | 23 Y / 20 N | 63.0% |
| noul-v2 LM Y/N | 75.5% | 82.1% | 81.4% | 3 Y / 40 N | 59.3% |
| noul-v2 binary head | 46.9% | 82.1% | 16.3% | 43/43 Y | 59.3% |
| Constant-N | — | — | 83.7% | 43/43 N | — |
| ZS Qwen2.5-1.5B | 42.9% | 60.7% | 41.9% | — | 25.9% |
| ZS Qwen2.5-0.5B | 21.4% | 32.1% | 16.3% | all-Y | 18.5% |
Gold Noul is N-heavy (7 Y / 36 N), so always-N gets 36/43. v2 gets 35/43.
| Check | Outcome |
|---|---|
| Lift Noul vs noul-bin | yes (44.2% → 81.4%) |
| Choice around ≥80% | yes (82.1%) |
| Noul not all-Y | yes (3 Y / 40 N) |
| Beat constant-N | no (81.4% vs 83.7%) |
| No logit bias trick | yes (bias=0) |
codes / code_to_key; end with Answer:\n
so Y/N stay single trailing tokens.Do not default to the binary head. Optional ablation only:
OSS_JEV_NOUL_BINARY_HEAD=1 with noul_binary_head.pt.
python
# LM Y/N serving (what you want)codes, keys = resolve_codes_and_keys(question)# prompt must end with "Answer:\n"logits = model(...).logits[0, -1, code_token_ids]probs = softmax(logits)pred = codes[probs.argmax()]Default noul_logit_bias_n=0.
Live demo:
gyanbhambhani/oss-jev-decision-gradio
(this tip @ cfd24b3…; binary head off).
Answer:\n lockstepWithout the newline, Qwen2.5 merges Answer:N into :N while
Answer:Y splits cleanly — that historically locked all-Y. Train and
infer both need Answer:\n.
The Linear(h→2) aux CE helped move the LM surface. Serving the head itself stays 43/43 Y. Keep the head for training; serve LM.
81.4% Noul beats the old all-Y floor (16.3%) and noul-bin (44.2%), but constant-N (83.7%) still wins by one frozen-98 row. Prefer live Jev when Noul competence is the product bar.
N=98 → wide CIs. Treat ±~10 pp overall as noise.
Continue Job from noul-bin (Path A). New Hub id — did not overwrite
…-decision / …-10k / …-noul-aug / …-noul-bin.
text
model_id: …-decision-noul-binsoft_ce + constrained_codes + balance_noulnoul_n_frac=0.65 # was 0.50noul_mix_weight=3.0 # NEW — upsample noul vs choice/scorenoul_random_codes + noul_negation_pairs + cons_w=1.0noul_binary_head=True (weight 1.0)noul_binary_balanced=True # NEW — class-balanced binary CEnoul_logit_bias_n=0.0max_steps=4000 # was 2000lr=1e-5 bf16 a10g-smallAnswer:\n lockstepTrain-src v0.1.7 adds noul_mix_weight, noul_binary_balanced,
and warm-start of noul_binary_head.pt from the continue-from Hub tip.
| Continue-from | gyanbhambhani/oss-jev-qwen05-decision-noul-bin |
| Base lineage | Qwen2.5-0.5B-Instruct → 10k → noul-aug → noul-bin → v2 |
| Mode | one_token |
| Max length | 1024 |
| Batch size | 2 |
| Learning rate | 1e-5 |
| Max steps | 4000 |
| Seed | 42 |
| Precision | bf16 (fp16 off) |
| Optimizer | AdamW fused (β=(0.9, 0.999), ε=1e-8) |
| LR schedule | linear |
| Logging | every 50 steps |
| Soft CE | on |
| Constrained codes | on |
| balance_noul | on (noul_n_frac=0.65) |
noul_mix_weight | 3.0 |
| Negation / random codes | on (cons_w=1.0) |
| Binary head | train aux, weight 1.0, class-balanced |
| Logit bias N | 0.0 |
| Train-src | gyanbhambhani/oss-jev-train-src v0.1.7 |
| Hardware | HF Jobs a10g-small, ~35 min |
| Config | configs/jobs-a10g-noul-v2.yaml |
| Job | 6ad2748b5ccb47ba0f7b4f1e |
text
@misc{oss-jev-qwen05-decision-noul-v2, title={OSS Jev Noul v2 continue (0.5B)}, author={Bhambhani, Gyan}, year={2026}, howpublished={Hugging Face}, url={https://huggingface.co/gyanbhambhani/oss-jev-qwen05-decision-noul-v2}}# LM Y/N serving (what you want)codes, keys = resolve_codes_and_keys(question)# prompt must end with "Answer:\n"logits = model(...).logits[0, -1, code_token_ids]probs = softmax(logits)pred = codes[probs.argmax()]model_id: …-decision-noul-binsoft_ce + constrained_codes + balance_noulnoul_n_frac=0.65 # was 0.50noul_mix_weight=3.0 # NEW — upsample noul vs choice/scorenoul_random_codes + noul_negation_pairs + cons_w=1.0noul_binary_head=True (weight 1.0)noul_binary_balanced=True # NEW — class-balanced binary CEnoul_logit_bias_n=0.0max_steps=4000 # was 2000lr=1e-5 bf16 a10g-smallAnswer:\n lockstep@misc{oss-jev-qwen05-decision-noul-v2, title={OSS Jev Noul v2 continue (0.5B)}, author={Bhambhani, Gyan}, year={2026}, howpublished={Hugging Face}, url={https://huggingface.co/gyanbhambhani/oss-jev-qwen05-decision-noul-v2}}