GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
ritwikraha
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ritwikraha
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
Qwen/Qwen3.5-4Bmergedtinker://e45e676a-8586-5d5d-a00f-2c4527c33929:train:0/sampler_weights/finalqwen3_5_disable_thinkingritwikraha/calibrate-qwen-curatedThe selected checkpoint delivered the strongest general-purpose result in the core report. The primary in-domain and OOD splits each contain 250 evaluated examples. These are release-candidate measurements rather than complete benchmark estimates when the report scope is core.
| Metric | Base | Off-policy | Unified on-policy |
|---|---|---|---|
| In-domain accuracy | 0.8800 | 0.8720 | 0.8840 |
| Structured-output validity | 0.7080 | 0.8400 | 0.9320 |
| Verbal-confidence ECE | 0.2314 | 0.2232 | 0.1182 |
| Multiclass Brier score | 0.1891 | 0.2000 | 0.1831 |
OOD accuracy is 0.7600, OOD multiclass NLL is 0.8066, and OOD AURC is 0.1014. Option-order choice-text consistency is 0.8940.
Use the following system instruction:
text
Answer the multiple-choice question. Return one JSON object with exactly the keys "answer", "confidence", "justification", and "abstain". answer must be one uppercase option label from the prompt. confidence must be a number between 0 and 1. justification must be one concise sentence. abstain must be a JSON boolean.
Expected response:
json
{"answer":"B","confidence":0.91,"justification":"The evidence directly supports option B.","abstain":false}
For a merged release:
python
from transformers import AutoModelForCausalLM, AutoTokenizertokenizer = AutoTokenizer.from_pretrained("ritwikraha/calibrate-qwen-unified")model = AutoModelForCausalLM.from_pretrained("ritwikraha/calibrate-qwen-unified",device_map="auto",torch_dtype="auto",trust_remote_code=True,)
For the PEFT adapter release:
python
from peft import PeftModelfrom transformers import AutoModelForCausalLMbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", device_map="auto")model = PeftModel.from_pretrained(base, "ritwikraha/calibrate-qwen-unified-adapter")
Merged model:
bash
vllm serve ritwikraha/calibrate-qwen-unified
PEFT adapter:
bash
vllm serve Qwen/Qwen3.5-4B --enable-lora --lora-modules calibrate=ritwikraha/calibrate-qwen-unified-adapter
The validation-fitted answer-probability temperature is 1.082009 and the 80 percent target-coverage threshold is 0.890585. Load calibration_config.json and use calibration.py when the serving stack scores every option label. The generated verbal confidence remains available when option scoring is unavailable.
The release targets English multiple-choice prompts. Calibration was fitted on the curated validation mixture and can shift under new domains, prompt templates, quantization, or decoding settings. The provided evaluation snapshot uses 250 examples per primary split. Refit calibration after material deployment changes and use a full evaluation before making final benchmark claims.
Training, dataset curation, evaluation, and publication notebooks are available in AutoRegressive-Bhasha. The model repository includes the frozen calibration policy, output schema, evaluation summary, and compact report evidence.
| Multiclass NLL | 0.3811 | 0.4051 | 0.3686 |
| Selective accuracy | 0.9593 | 0.9497 | 0.9674 |