GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
dustarrr
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
dustarrr
Model Tree
Qwen/Qwen2.5-1.5B
Input Modalities
Output Modalities
Supported Functionality
| Base model | Qwen/Qwen2.5-1.5B |
| Parameters | ~1.5B (LoRA r=16, merged) |
| Context length | 2048 tokens |
| Training data | s1K (1,000 traces) + LIMO (817 traces) = ~1,800 CoT samples |
| Method | s1-style distillation + budget forcing via QLoRA SFT |
| Compute | Google Colab T4 GPU, ~16 min |
| Special tokens | <think> </think> for reasoning trace delimiters |
| Benchmark | Reasoning Rob |
|---|---|
| GSM8K (50 samples) | 10.00% |
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "dustarrr/reasoning-rob", torch_dtype=torch.float16, device_map="auto",)tokenizer = AutoTokenizer.from_pretrained("dustarrr/reasoning-rob")model.eval() messages = [ {"role": "system", "content": "You are a helpful assistant that thinks step by step."}, {"role": "user", "content": "If a train travels 60 km in 1.5 hours, what is its speed?"},]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt") with torch.no_grad(): outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False) response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)print(response)Extend the model's thinking phase by injecting "Wait" before the </think> token
to force longer reasoning before the final answer. This is the test-time scaling
trick from the s1 paper.
| Hyperparameter | Value |
|---|---|
| LoRA rank | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Learning rate | 0.0001 |
| LR scheduler | cosine |
| Warmup ratio | 0.03 |
| Weight decay | 0.01 |
| Batch size | 2 |
| Gradient accumulation | 8 |
Reasoning Rob is a QLoRA fine-tune of Qwen/Qwen2.5-1.5B (base, not instruct) trained on:
Using the s1 distillation + budget-forcing method and LIMO "less is more" reasoning transfer approach.
All credit to:
This model would not exist without their work.
Apache 2.0 (inherited from Qwen2.5 base model).
Generated on 2026-06-23
| Max sequence length | 2048 |
| Epochs | 1 |
| Quantization | NF4 (4-bit, double quant) |
| Optimizer | adamw_torch |