GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
abderahmenn
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
abderahmenn
Model Tree
Qwen/Qwen3.5-9B
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-9B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "abderahmenn/qwen3.5-9b-sob-json-lora")tok = AutoTokenizer.from_pretrained("abderahmenn/qwen3.5-9b-sob-json-lora")Prompts follow the training data's chat format, with thinking disabled:
python
messages = [ {"role": "system", "content": "You extract structured JSON from text. Given a SCHEMA, CONTEXT, and QUESTION, return a single valid JSON object that matches the schema and is grounded in the context. No markdown, no prose."}, {"role": "user", "content": f"Schema:\n{schema}\n\nContext:\n{context}\n\nQuestion: {question}\n\n" "Return ONLY valid JSON that matches the schema, using information from the context. No explanation."},]prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)Keep it in bf16: Qwen3.5 is unusually sensitive to quantisation. If you quantise, benchmark it first.
| Base model | Qwen/Qwen3.5-9B |
| Data | MoetezF/Finetuning-ready (extract + repair), stratified 80/10/10 split on task × source |
| Train / eval rows | 19,999 / 200 |
| Steps | 1250 of 1250 (epoch 1.00) |
| LoRA | r=32, alpha=32, dropout=0.05 |
| Target modules | down_proj, gate_proj, in_proj_qkv, in_proj_z, k_proj, o_proj, out_proj, q_proj, up_proj, v_proj |
| Optimiser | AdamW, lr 1e-04, cosine schedule, 37 warmup steps |
| Effective batch | 16 (batch 1 × grad-accum 16), max length 4096 |
| Precision | bf16, loss on the assistant turn only |
| Hardware | AMD Ryzen AI Max+ 395 (Strix Halo, gfx1151), ROCm |
| Eval loss | 0.1432 (first eval) → 0.1106 (best, checkpoint-1250, the one shipped here) |
DS1 — interfaze-ai/sob test (generation) (%, higher is better except over_correction)
| model | subset | n | json_pass | value_accuracy | faithfulness | path_recall | structure_coverage | type_safety | perfect_response |
|---|---|---|---|---|---|---|---|---|---|
| base | full | 5,000 | 99.4 | 81.3 | 86.9 | 98.5 | 98.5 | 99.9 | 48.0 |
| base | clean | 4,568 | 99.4 | 81.3 | 87.0 | 98.5 | 98.5 | 99.9 | 47.9 |
| ft | full | 5,000 | 99.6 | 84.1 | 89.0 | 98.8 | 98.9 | 99.9 | 54.2 |
| ft | clean | 4,568 | 99.6 | 84.1 | 89.1 | 98.8 | 98.9 | 99.9 | 54.0 |
DS2 — seneca-center/SOB-with-errors-injection (verify/repair) (%, higher is better except over_correction)
| model | subset | n | detection_correct | localization_ok | repair_exact | schema_ok | over_correction |
|---|---|---|---|---|---|---|---|
| base | full | 6,667 | 72.3 | 58.6 | 55.8 | 87.6 | 59.3 |
| base | clean | 6,100 | 72.2 | 58.1 | 55.8 | 87.7 | 59.1 |
| ft | full | 6,667 | 90.4 | 92.5 | 77.6 | 98.9 | 33.1 |
| ft | clean | 6,100 | 90.4 | 92.6 | 77.8 | 98.9 | 32.7 |
Contamination. 430 DS1 and 430 DS2 source records share a context passage with the training split (SOB's text split is HotpotQA-derived, as is part of the training corpus); none share a record id. clean rows exclude them; prefer those numbers.
base is Qwen/Qwen3.5-9B with identical prompts; both use greedy decoding with thinking disabled.
DS1 metrics follow SOB (arXiv:2604.25359): value accuracy and faithfulness are gated on a parseable,
schema-valid root and on path-F1 ≥ 0.95. DS2 mixes errored candidates with 25% clean ones, so
over_correction (editing an already-correct answer) is measured on the clean candidates only.
import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-9B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "abderahmenn/qwen3.5-9b-sob-json-lora")tok = AutoTokenizer.from_pretrained("abderahmenn/qwen3.5-9b-sob-json-lora")messages = [ {"role": "system", "content": "You extract structured JSON from text. Given a SCHEMA, CONTEXT, and QUESTION, return a single valid JSON object that matches the schema and is grounded in the context. No markdown, no prose."}, {"role": "user", "content": f"Schema:\n{schema}\n\nContext:\n{context}\n\nQuestion: {question}\n\n" "Return ONLY valid JSON that matches the schema, using information from the context. No explanation."},]prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)