GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
jsaurabh
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
jsaurabh
Model Tree
Qwen/Qwen3.5-9B
Input Modalities
Output Modalities
Supported Functionality
| Evaluation | Result |
|---|---|
| Nimble frozen holdout | 282/324 (87.04%) |
| Synthetic gap holdout | 420/480 (87.50%) |
| JevBench public v1 | 184/231 (79.65%) |
| JevBench Brier score | 0.2895 |
| JevBench ECE | 0.0892 |
JevBench tiers: easy 48/48, standard 68/72, hard 68/111. This run matched the original reproduced Nimble adapter on public JevBench accuracy; it improved calibration error but did not improve accuracy.
c202236235762e1c871ad0ccb60c8ee5ba337b9aThe exact configuration and data audit are in schema_config.json and data_audit.json. run_report.json, after_original.json, after_gap.json, and nimble-data-mix-v2-jevbench-public.json contain the evaluation evidence.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base_id = "Qwen/Qwen3.5-9B"adapter_id = "jsaurabh/qwen3.5-9b-jev-data-mix-v2" tokenizer = AutoTokenizer.from_pretrained(adapter_id)base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto")model = PeftModel.from_pretrained(base, adapter_id).eval()The adapter was trained and evaluated with the candidate-scoring implementation from Bespoke Labs' Nimble repository, frozen at commit f136b3f75721fda4ea961f73993cc50b08488835. Normal text generation does not reproduce the reported scores; use Nimble's schema encoding and candidate-logit readout.
For the official JevBench runner, see bench/SUBMISSION.md. The included in-process adapter performs the same frozen candidate-logit scoring path and generates no output tokens.
f136b3f75721fda4ea961f73993cc50b084888355e95f23cbb7be098a9061fea924c4421620ab1a5c43828fe7a6005682754878a21b03eebd167cb745e212af81277757a262b903bThis is a research adapter, not a production banking or compliance model. Its weakest public benchmark area is temporal/numeric reasoning. The synthetic gap set is private project-generated evaluation data and should not be interpreted as an independent benchmark. Review decisions in high-impact settings and validate the adapter on the intended distribution.
The original Nimble training projection was public. The added synthetic curriculum was created after inspecting this project's results on the public JevBench tasks and targeted broad weak families including temporal/numeric, probability, and long-policy reasoning. Public JevBench performance was used to compare experiments. No private or held-out JevBench item was available or used, and the adapter was not calibrated on JevBench probabilities. This is therefore a benchmark-aware submission, not a clean blind evaluation.
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base_id = "Qwen/Qwen3.5-9B"adapter_id = "jsaurabh/qwen3.5-9b-jev-data-mix-v2" tokenizer = AutoTokenizer.from_pretrained(adapter_id)base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto")model = PeftModel.from_pretrained(base, adapter_id).eval()