Qwen3.5-0.8B-SFT-Claude-Reasoning API & Inference Endpoint

Overview

Table with columns: Property, Value
Property	Value
Developed by	Behrooz Azarkhalili
License	APACHE-2.0
Language	English
Base Model	Qwen/Qwen3.5-0.8B
Model Size	0.8B parameters
Training Method	SFT with LoRA (TRL)
Dataset	Claude Opus Reasoning Distillation (~10,477 samples)
Context Length	2,048 tokens
Hardware	NVIDIA H100 80GB HBM3 (MIG 3g.40gb)
Cluster	DRAC / Fir (Compute Canada)

Training Configuration

Table with columns: Parameter, Value
Parameter	Value
Framework	TRL
Learning Rate	2e-4
Batch Size	2 per device
Gradient Accumulation	8
Effective Batch Size	16
Training	1 epoch(s)
Max Sequence Length	2,048 tokens
Precision	BF16 mixed precision
Gradient Checkpointing	Enabled

LoRA Configuration

Table with columns: Parameter, Value
Parameter	Value
LoRA Rank (r)	64
LoRA Alpha	128
LoRA Dropout	0.05
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Quantization	4-bit NF4 (QLoRA)

Usage

Quick Start

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain step by step how to solve 2x + 5 = 13."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Pipeline

python
from transformers import pipeline
generator = pipeline("text-generation", model="ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning", device_map="auto")
output = generator([{"role": "user", "content": "What is 2+2?"}], max_new_tokens=256, return_full_text=False)
print(output[0]["generated_text"])

4-bit Quantized

python
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch
model = AutoModelForCausalLM.from_pretrained(
    "ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16),
    device_map="auto"
)

GGUF Versions

Quantized GGUF versions for CPU/edge inference: ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning-GGUF

Table with columns: Quantization, Bits, Use Case
Quantization	Bits	Use Case
Q2_K	2	Edge devices, mobile
Q3_K_M	3	Constrained environments
Q4_K_M	4	Best quality/size balance (recommended)
Q5_K_M	5	Higher quality
Q6_K	6	Near-lossless
Q8_0	8

Ollama

bash
ollama pull hf.co/ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning-GGUF:Q4_K_M
ollama run hf.co/ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning-GGUF:Q4_K_M "Hello!"

llama.cpp

bash
llama-cli -m Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning-Q4_K_M.gguf -p "Your prompt here" -n 256

Limitations

Language: Primarily trained on English data
Knowledge Cutoff: Limited to base model's training data cutoff
Hallucinations: May generate plausible-sounding but incorrect information
Context Length: Fine-tuned with 2,048 token limit
Safety: Not extensively safety-tuned; use with appropriate guardrails

Intended Use

Research on language model fine-tuning and reasoning distillation
Educational purposes and experimentation
Prototyping conversational AI and tool-use agents
Not recommended for production without additional safety measures

Framework Versions

TRL: 1.0.0
Transformers: 5.5.3
PyTorch: 2.11.0
PEFT: 0.18.1
BitsAndBytes: 0.49.2

Citation

bibtex
@misc{azarkhalili2026qwen35_08b_sft_claude_reasoning,
    author = {Azarkhalili, Behrooz},
    title = {Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning: Fine-tuned Qwen3.5-0.8B for SFT Distillation},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning}
}

To generate a citable DOI, click "Cite this model" on the model page.

Acknowledgments

Base model developers (Qwen)
Hugging Face TRL Team
Salesforce xLAM for the function calling dataset
Compute Canada / DRAC for HPC resources

Property

Value

Developed by

Behrooz Azarkhalili

License

APACHE-2.0

Language

English

Base Model

Qwen/Qwen3.5-0.8B

Model Size

0.8B parameters

Training Method

SFT with LoRA (TRL)

Dataset

Claude Opus Reasoning Distillation (~10,477 samples)

Context Length

2,048 tokens

Hardware

NVIDIA H100 80GB HBM3 (MIG 3g.40gb)

Cluster

DRAC / Fir (Compute Canada)

Parameter

Value

Framework

TRL

Learning Rate

2e-4

Batch Size

2 per device

Gradient Accumulation

Effective Batch Size

Training

1 epoch(s)

Max Sequence Length

2,048 tokens

Precision

BF16 mixed precision

Gradient Checkpointing

Enabled

Parameter

Value

LoRA Rank (r)

LoRA Alpha

128

LoRA Dropout

0.05

Target Modules

q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Quantization

4-bit NF4 (QLoRA)

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain step by step how to solve 2x + 5 = 13."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

python

from transformers import pipeline
generator = pipeline("text-generation", model="ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning", device_map="auto")
output = generator([{"role": "user", "content": "What is 2+2?"}], max_new_tokens=256, return_full_text=False)
print(output[0]["generated_text"])

python

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch
model = AutoModelForCausalLM.from_pretrained(
    "ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16),
    device_map="auto"
)

Quantization

Bits

Use Case

Q2_K

Edge devices, mobile

Q3_K_M

Constrained environments

Q4_K_M

Best quality/size balance (recommended)

Q5_K_M

Higher quality

Q6_K

Near-lossless

Q8_0

bibtex

@misc{azarkhalili2026qwen35_08b_sft_claude_reasoning,
    author = {Azarkhalili, Behrooz},
    title = {Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning: Fine-tuned Qwen3.5-0.8B for SFT Distillation},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/ermiaazarkhalili/Qwen3.5-0.8B-SFT-Claude-Opus-Reasoning}
}

Qwen3.5-0.8B-SFT-Claude-Reasoning

README

Overview

Training Configuration

LoRA Configuration

Usage

Quick Start

Pipeline

4-bit Quantized

GGUF Versions

Ollama

llama.cpp

Limitations

Intended Use

Framework Versions

Citation

Acknowledgments

Explore FriendliAI today

README

Overview

Training Configuration

LoRA Configuration

Usage

Quick Start

Pipeline

4-bit Quantized

GGUF Versions

Ollama

llama.cpp

Limitations

Intended Use

Framework Versions

Citation

Acknowledgments