Training and merge details
- Training method: GRPO with QLoRA/LoRA.
- Adapter source:
qwen36-reasoning-effort-grpo-0.1-qlora-200-12k/checkpoint-200.
- Training steps: 200.
- LoRA rank: 32.
- LoRA alpha: 64.
- LoRA dropout: 0.0.
- Target modules:
q_proj, k_proj, v_proj, o_proj, shared_expert.gate_proj, shared_expert.up_proj, shared_expert.down_proj.
- Trainable token IDs: 248077, 248078, 248079.
- Base precision during merge: bfloat16.
- Published weights: safetensors, split into 15 shards.
- Merge operation: PEFT
merge_and_unload().
The original adapter artifacts are preserved separately from this merged release. The published safetensors use the canonical Qwen 3.6 packed-MoE layout required by current inference runtimes.
Reasoning effort interface
The tokenizer includes three project-specific special tokens:
<|reasoning_effort_low|> (token ID 248077)
<|reasoning_effort_medium|> (token ID 248078)
<|reasoning_effort_high|> (token ID 248079)
The published chat template accepts reasoning_effort="low", "medium", or "high" and emits the corresponding token before the thinking block. No xhigh token is provided.
Example:
from transformers import AutoModelForImageTextToText, AutoTokenizer
model_id = "valendra/sherry-35b-a3b-0.1-grpo-preview"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
model_id,
device_map="auto",
dtype="auto",
)
inputs = tokenizer.apply_chat_template(
[{"role": "user", "content": "Solve: 12 * 13"}],
add_generation_prompt=True,
enable_thinking=True,
reasoning_effort="medium",
return_tensors="pt",
).to(model.device)
outputs = model.generate(inputs, max_new_tokens=2048)
print(tokenizer.decode(outputs[0, inputs.shape[-1]:], skip_special_tokens=False))
For non-thinking answers, call the template with enable_thinking=False. The model is multimodal-compatible through the Qwen3.6 architecture, but this release was trained and smoke-tested primarily through its text-generation path.
Limitations
- Preview release; independent quality evaluation is incomplete.
- The logged training reward is not an accuracy or benchmark score.
- Generation length and quality may vary by reasoning-effort level and prompt.
- Use the base model's license and usage requirements.
Intended use
Research and experimentation with controllable reasoning effort. Validate outputs for your use case before deployment.