clzoro

Qwen3.6-35B-A3B-Claude-Distill-v2

Deploy Dedicated

README

License: apache-2.0

Model Description

Base model: Qwen/Qwen3.6-35B-A3B (MoE, 256 experts, 3B active parameters)
Training data: Kassadin88/Claude-Distills
Training method: Full SFT (all parameters)
Training framework: ms-swift
Training hardware: 2 nodes × 8× H100 80GB

Training Data

The model was trained on 125,175 Claude-distilled conversation pairs (with 6,588 held out for validation).

Data Distribution

Categories were classified using an automated LLM-based classifier on a representative sample:

Table with columns: Category, Percentage
Category	Percentage
Math	65.5%
Code	15.1%
Knowledge	5.1%
Science	5.0%
Conversation	2.6%
Reasoning	2.1%
Writing	1.4%
Instruction	1.4%
Other	1.8%

The dataset is dominated by math and code (~80%), with significant coverage of science, knowledge, and general conversation.

Data Format

Each sample is a multi-turn conversation in standard chat format:

json

{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

Benchmark Results

Base model (Qwen3.6-35B-A3B) performance on language benchmarks:

Table with columns: Benchmark, Score
Benchmark	Score
MMLU-Pro	85.2
MMLU-Redux	93.3
C-Eval	90.0
GPQA	86.0
SuperGPQA	64.7
HLE	21.4
LiveCodeBench v6	80.4
HMMT Feb 25	90.7
HMMT Nov 25	89.1

For full benchmark details including vision-language results, see the base model card.

Quickstart

Note: This model operates in thinking mode by default, generating thinking content signified by thinking\n... before producing the final response. To disable thinking, refer to Non-Thinking Mode.

Hugging Face Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2" tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(    model_name,    torch_dtype="auto",    device_map="auto",) messages = [    {"role": "user", "content": "Write a Python function to find the longest common subsequence of two strings."}] text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=32768)response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response)

OpenAI-Compatible API (via vLLM / SGLang)

shell

# vLLMvllm serve Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2 --port 8000 --tensor-parallel-size 4 --max-model-len 262144 --reasoning-parser qwen3 # SGLangpython -m sglang.launch_server --model-path Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2 --port 8000 --tp-size 4 --mem-fraction-static 0.8 --context-length 262144 --reasoning-parser qwen3

python

from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") messages = [    {"role": "user", "content": "Solve: find all integer solutions to x^2 + y^2 = 25"}] chat_response = client.chat.completions.create(    model="Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2",    messages=messages,    max_tokens=32768,    temperature=1.0,    top_p=0.95,    presence_penalty=1.5,    extra_body={"top_k": 20},)print(chat_response.choices[0].message.content)

Non-Thinking Mode

To get direct responses without chain-of-thought reasoning:

python

text = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,    enable_thinking=False,)

Or via API:

python

chat_response = client.chat.completions.create(    model="Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2",    messages=messages,    max_tokens=32768,    temperature=0.7,    top_p=0.8,    presence_penalty=1.5,    extra_body={        "top_k": 20,        "chat_template_kwargs": {"enable_thinking": False},    },)

Recommended Sampling Parameters

Table with columns: Mode & Task, temperature, top_p, top_k, presence_penalty
Mode & Task	temperature	top_p	top_k	presence_penalty
Thinking - general tasks	1.0	0.95	20	1.5
Thinking - precise coding	0.6	0.95	20	0.0
Non-thinking - general tasks	0.7	0.8	20	1.5

Training Details

Table with columns: Item, Value
Item	Value
Training epochs	2
Global steps	1,956
Final train loss	0.479
Final eval loss	0.579
DeepSpeed	ZeRO-3 Offload
Precision	BF16
Gradient checkpointing	Enabled
Nodes	2 × 8 GPU

Best Practices

Adequate Output Length: Use at least 32,768 tokens for output. For complex math/coding problems, set max output length to 81,920 tokens.
No Thinking Content in History: In multi-turn conversations, historical model output should only include the final response, not thinking content. The provided chat template handles this automatically.
Standardize Output Format: For math problems, include "Please reason step by step, and put your final answer within \boxed{}." in the prompt.

Limitations

The model is primarily trained on English and Chinese data; performance on other languages may be limited.
Math and code dominate the training distribution (~80%), which may affect performance on other domains.
As a distilled model, it may inherit biases present in the Claude-generated training data.
The model has not been aligned with RLHF or similar safety techniques.

Citation

If you find our work helpful, feel free to give us a cite.

bibtex

@misc{qwen3.6,    title  = {{Qwen3.6}},    author = {{Qwen Team}},    year   = {2026},    url    = {https://qwen.ai/blog?id=qwen3.6}}

bibtex

@misc{qwen36-35b-a3b-claude-distill-v2,    title={Qwen3.6-35B-A3B-Claude-Distill-v2: Full SFT on Claude Distill Data},    author={Kassadin88},    year={2026},    url={https://huggingface.co/Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

clzoro

Model Tree

Base

Qwen/Qwen3.6-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Description

Base model: Qwen/Qwen3.6-35B-A3B (MoE, 256 experts, 3B active parameters)
Training data: Kassadin88/Claude-Distills
Training method: Full SFT (all parameters)
Training framework: ms-swift
Training hardware: 2 nodes × 8× H100 80GB

Training Data

The model was trained on 125,175 Claude-distilled conversation pairs (with 6,588 held out for validation).

Data Distribution

Categories were classified using an automated LLM-based classifier on a representative sample:

Table with columns: Category, Percentage
Category	Percentage
Math	65.5%
Code	15.1%
Knowledge	5.1%
Science	5.0%
Conversation	2.6%
Reasoning	2.1%
Writing	1.4%
Instruction	1.4%
Other	1.8%

The dataset is dominated by math and code (~80%), with significant coverage of science, knowledge, and general conversation.

Data Format

Each sample is a multi-turn conversation in standard chat format:

json

{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

Benchmark Results

Base model (Qwen3.6-35B-A3B) performance on language benchmarks:

Table with columns: Benchmark, Score
Benchmark	Score
MMLU-Pro	85.2
MMLU-Redux	93.3
C-Eval	90.0
GPQA	86.0
SuperGPQA	64.7
HLE	21.4
LiveCodeBench v6	80.4
HMMT Feb 25	90.7
HMMT Nov 25	89.1

For full benchmark details including vision-language results, see the base model card.

Quickstart

Note: This model operates in thinking mode by default, generating thinking content signified by thinking\n... before producing the final response. To disable thinking, refer to Non-Thinking Mode.

Hugging Face Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2" tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(    model_name,    torch_dtype="auto",    device_map="auto",) messages = [    {"role": "user", "content": "Write a Python function to find the longest common subsequence of two strings."}] text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=32768)response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response)

OpenAI-Compatible API (via vLLM / SGLang)

shell

# vLLMvllm serve Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2 --port 8000 --tensor-parallel-size 4 --max-model-len 262144 --reasoning-parser qwen3 # SGLangpython -m sglang.launch_server --model-path Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2 --port 8000 --tp-size 4 --mem-fraction-static 0.8 --context-length 262144 --reasoning-parser qwen3

python

from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") messages = [    {"role": "user", "content": "Solve: find all integer solutions to x^2 + y^2 = 25"}] chat_response = client.chat.completions.create(    model="Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2",    messages=messages,    max_tokens=32768,    temperature=1.0,    top_p=0.95,    presence_penalty=1.5,    extra_body={"top_k": 20},)print(chat_response.choices[0].message.content)

Non-Thinking Mode

To get direct responses without chain-of-thought reasoning:

python

text = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,    enable_thinking=False,)

Or via API:

python

chat_response = client.chat.completions.create(    model="Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2",    messages=messages,    max_tokens=32768,    temperature=0.7,    top_p=0.8,    presence_penalty=1.5,    extra_body={        "top_k": 20,        "chat_template_kwargs": {"enable_thinking": False},    },)

Recommended Sampling Parameters

Table with columns: Mode & Task, temperature, top_p, top_k, presence_penalty
Mode & Task	temperature	top_p	top_k	presence_penalty
Thinking - general tasks	1.0	0.95	20	1.5
Thinking - precise coding	0.6	0.95	20	0.0
Non-thinking - general tasks	0.7	0.8	20	1.5

Training Details

Table with columns: Item, Value
Item	Value
Training epochs	2
Global steps	1,956
Final train loss	0.479
Final eval loss	0.579
DeepSpeed	ZeRO-3 Offload
Precision	BF16
Gradient checkpointing	Enabled
Nodes	2 × 8 GPU

Best Practices

Adequate Output Length: Use at least 32,768 tokens for output. For complex math/coding problems, set max output length to 81,920 tokens.
No Thinking Content in History: In multi-turn conversations, historical model output should only include the final response, not thinking content. The provided chat template handles this automatically.
Standardize Output Format: For math problems, include "Please reason step by step, and put your final answer within \boxed{}." in the prompt.

Limitations

The model is primarily trained on English and Chinese data; performance on other languages may be limited.
Math and code dominate the training distribution (~80%), which may affect performance on other domains.
As a distilled model, it may inherit biases present in the Claude-generated training data.
The model has not been aligned with RLHF or similar safety techniques.

Citation

If you find our work helpful, feel free to give us a cite.

bibtex

@misc{qwen3.6,    title  = {{Qwen3.6}},    author = {{Qwen Team}},    year   = {2026},    url    = {https://qwen.ai/blog?id=qwen3.6}}

bibtex

@misc{qwen36-35b-a3b-claude-distill-v2,    title={Qwen3.6-35B-A3B-Claude-Distill-v2: Full SFT on Claude Distill Data},    author={Kassadin88},    year={2026},    url={https://huggingface.co/Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2}}