Model details
Table | |
|---|
| Developed by | Horizon Tech d.o.o., Sarajevo, Bosnia and Herzegovina |
| Model type | 27B-parameter vision-language model (text and images in, text out), with optional reasoning ("thinking") before the answer |
| Fine-tuned from | Qwen/Qwen3.8-27B, revision 1d4bf0f |
| Method | Rejection-sampling fine-tuning (RFT) with a LoRA adapter, merged into the weights |
| Weights | bfloat16 safetensors, 18 shards, 52 GB; retrained MTP head included (mtp.* tensors) |
| Languages | As the base model; evaluated in English and German |
| License | Apache 2.0 |
| Released | November 2026 |
| Author | Hamza Abdagic (ORCID 0009-0002-4078-4922) |
| Contact | hamza.abdagic@horizon-tech.io |
Versions
What was done
- Base: Qwen/Qwen3.8-27B, revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0.
- Method: the base model answered training problems itself, programs graded the answers, and the model was trained on its own correct, shortest answers. No teacher model and no model-written grading.
- Training: LoRA (rank 64, alpha 128), 1.5 epochs, learning rate 1e-4, sequences packed to 32,768 tokens. Validation loss went from 0.283 to 0.234.
- These weights: the adapter merged into the base in fp32 and stored as bfloat16. The vision tower is unchanged.
Evaluation
Paired comparison against the base on the same items (McNemar test). No difference is significant (smallest p = 0.12 with thinking on).
Table with columns: Benchmark, n, Base, thinking on, RFT v1, thinking on, Base, thinking off, RFT v1, thinking off| Benchmark | n | Base, thinking on | RFT v1, thinking on | Base, thinking off | RFT v1, thinking off |
|---|
| GSM8K | 1319 | 97.95 | 97.88 | 96.59 | 96.82 |
| MMLU-Pro (subset) | 200 | 86.50 | 84.00 | 65.50 | 65.50 |
| MATH-500 | 500 | 98.20 | 98.40 | 97.40 | 98.20 |
| AIME 2025 | 30 | 83.33 | 83.33 | 70.00 | 63.33 |
| HumanEval | 164 | 93.90 | 95.73 | 97.56 | 97.56 |
| MBPP | 500 | 94.20 | 93.80 | 81.00 | 80.80 |
| IFEval | 541 | 89.09 | 90.76 | 82.07 | 81.89 |
| German MMLU (subset) | 200 | 83.50 | 86.50 | 80.00 | 81.00 |
Reasoning tokens per correct answer, thinking on (RFT v1 vs base, same run as the table):
Table with columns: Benchmark, Base, RFT v1, Change| Benchmark | Base | RFT v1 | Change |
|---|
| GSM8K | 389 | 338 | −13% |
| MMLU-Pro | 1,702 | 1,179 | −31% |
| MATH-500 | 1,481 | 1,328 | −10% |
| AIME 2025 | 9,018 | 8,845 | −2% |
| HumanEval | 1,531 | 1,070 | −30% |
| MBPP | 2,885 | 2,603 | −10% |
The number of answers cut off at the token limit did not change in a consistent direction.
Limitations
- This is not a more accurate model than the base. Several of these benchmarks are close to their ceiling for the base, so they cannot show a gain.
- The multi-token-prediction (MTP) head was retrained for this model by self-distillation (the main weights are unchanged by it). Measured on 120 held-out documents, the retrained head gets 3.15 drafted tokens accepted at 3 drafts and 4.06 at 5 drafts; the head copied from the base got 3.02 and 3.63. These are offline numbers, not serving speed.
- Scores come from our own harness and are meant for the base-vs-RFT comparison, not for comparison with numbers published elsewhere.
How to use
Same usage as the base model (Transformers, vLLM, SGLang); the chat template and thinking controls are unchanged.
import torchfrom transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration repo = "ghostrider0470/Qwen3.8-27B-RFT-v1"tok = AutoTokenizer.from_pretrained(repo)model = Qwen3_5ForConditionalGeneration.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "A train leaves at 14:10 and arrives at 16:45. How long is the trip?"}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=True) # False = answer without reasoningids = tok(prompt, return_tensors="pt").to(model.device)out = model.generate(**ids, max_new_tokens=2048)print(tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))
vLLM
vllm serve ghostrider0470/Qwen3.8-27B-RFT-v1
Intended use
- A drop-in replacement for Qwen3.8-27B where reasoning length matters: the same answers for fewer reasoning tokens, so lower latency and cost per answer with thinking on.
- A starting point for further fine-tuning.
- It has the same safety behaviour and the same limits as the base model. It was not trained on any user or customer data.
Citation
@misc{horizontech2026rftv1, author = {Abdagic, Hamza}, title = {Qwen3.8-27B-RFT-v1}, year = {2026}, publisher = {Hugging Face}, organization = {Horizon Tech d.o.o., Sarajevo}, howpublished = {\url{https://huggingface.co/ghostrider0470/Qwen3.8-27B-RFT-v1}}}
Acknowledgements
License
Apache 2.0, as the base model.