DiogenesChen122

Qwen3.6-27B-Lora-20260817

Deploy Dedicated

README

License: apache-2.0

What this adapter is

Format: PEFT LoRA (~934 MB)
Rank / α: 64 / 128
Target modules (12): q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (dense + MLP) and in_proj_qkv, in_proj_z, in_proj_a, in_proj_b, out_proj (linear-attention layers of Qwen3.5/3.6's hybrid stack). Vision tower is deliberately excluded — the adapter lands only on model.language_model.*.
base_model_name_or_path: Qwen/Qwen3.6-27B (untied embeddings, 262k-vocab tokenizer, 262k context).

Data

Teacher: GPT-5.6, tree-search runs on 11 SuiteSparse matrices × 6 ops.
Selection: rejection-sampling SFT — keep a turn iff the kernel compiled, verified numerically, AND beat cuSPARSE by ≥1.5×. 172 kept out of 941 mined turns (≈2.4 M tokens/epoch).
Op mix (172 samples): 88 spmm_k8 · 70 spmm_k32 · 7 spmv · 5 spgemm · 1 spmm_k128 · 1 spmm_k256. Skewed toward small-K SpMM because that is where large speedups were easy to find; a model trained here will be much stronger on SpMM than on SpMV or SpGEMM.
Full statistics: training/distill/data/DATASET_CARD.md in the Dr.Sparse repo.

Training

Table

framework	`verl` 0.8.0, SFT trainer
precision	bfloat16 (frozen base + bf16 compute; LoRA in fp32)
optimizer	AdamW, LR 1e-5, cosine, 3 % warmup, weight decay 0
epochs / steps	3 / 60 total optimizer steps
global batch	8 (dynamic sequence packing)
max_length	24 576 (samples above this dropped at build time)
loss mask	assistant-turn only (pre-flight asserted)

Val loss (held-out by run_id so near-identical prompts don't straddle train/val): epoch 1 = 0.0316 → epoch 2 = 0.0302 → epoch 3 = 0.0298. Curve saturated by epoch 3.

Loading

python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3.6-27B"
lora_id = "DiogenesChen122/Qwen3.6-27B-Lora-20260817"

tok = AutoTokenizer.from_pretrained(lora_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16,
    device_map="auto", trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, lora_id)

Or serve via vLLM after merging:

bash
CUDA_VISIBLE_DEVICES=0,1 vllm serve <merged-dir> \
  --served-model-name Qwen/Qwen3.6-27B \
  --tensor-parallel-size 2 --dtype bfloat16 \
  --language-model-only --trust-remote-code \
  --max-model-len 32768 --port 8027

Coder system prompt

The adapter was trained with the exact system turn the Dr.Sparse coding agent sends at inference time (nodes/separate_file_coding_agent.py):

markdown
You are a CUDA sparse kernel specialist.

Chat template is applied with enable_thinking=False, matching the way the Dr.Sparse eval pipeline drives the model.

Caveats

SpMM-heavy. Op mix collapses to small-K SpMM at the 1.5× threshold — 92 % of samples. Expect stronger SpMM than SpMV or SpGEMM.
No teacher reasoning traces. GPT-5.6's Responses API returned an empty reasoning field for all 941 dumps, so the student is trained non-thinking.
Single GPU generation (H200) in the teacher runs. GPU-specific advice baked into prompts (SM count, L2 size) is what the student was trained on; evaluation on a different card is extrapolation.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

DiogenesChen122

Model Tree

Base

Qwen/Qwen3.6-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality