DiogenesChen122

Qwen3.8-27B-Lora-20260826

Deploy Dedicated

README

License: apache-2.0

Adapter

Format: PEFT LoRA (~934 MB)
Rank / α: 64 / 128
Target modules (12):
markdown
```
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
```
(dense + MLP) and
markdown
```
in_proj_qkv, in_proj_z, in_proj_a, in_proj_b, out_proj
```
(linear-attention layers of Qwen3.8's hybrid stack). Vision tower is excluded — the adapter lands only on model.language_model.*.
base_model_name_or_path: Qwen/Qwen3.8-27B.

Data

Teacher: GPT-5.6, tree-search runs across the entire Dr.Sparse evaluation corpus (level 1–4, multiple GPU generations).
Selection: rejection-sampling SFT — keep a turn iff the kernel compiled, verified numerically, and beat cuSPARSE by ≥ 1.05×. 4046 kept (3841 train / 205 val, held out by run_id) out of 5420 mined turns.
Op mix (4046 samples): 1023 spmm_k8 · 810 spmm_k32 · 435 spmm_k128 · 329 spmm_k256 · 753 spgemm · 696 spmv. Much better balance than the earlier 1.5× threshold — SpMV and SpGEMM are meaningfully represented now.
Corpus size: ~57 M tokens/epoch, med sequence 13.5 k tokens, max 22 k.

Training

Table

framework	`verl` 0.8.0, SFT trainer
precision	bfloat16 (frozen base + bf16 compute; LoRA in fp32)
optimizer	AdamW, LR 1e-5, cosine, 3 % warmup, weight decay 0
epochs / steps	3 / 2880 total optimizer steps
global batch	4 (dynamic sequence packing)
max_length	24 576 (samples above this dropped at build time)
loss mask	assistant-turn only (pre-flight asserted)

Val loss (best) = 0.02852 at the end of epoch 3. Training curves in the Dr.Sparse W&B project drsparse_distill.

Loading

python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3.8-27B"
lora_id = "DiogenesChen122/Qwen3.8-27B-Lora-20260826"

tok = AutoTokenizer.from_pretrained(lora_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16,
    device_map="auto", trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, lora_id)

Merge and serve via vLLM:

bash
CUDA_VISIBLE_DEVICES=0,1 vllm serve <merged-dir> \
  --served-model-name Qwen/Qwen3.8-27B \
  --tensor-parallel-size 2 --dtype bfloat16 \
  --language-model-only --trust-remote-code \
  --max-model-len 32768 --port 8028

Coder system prompt

The adapter was trained with the exact system turn the Dr.Sparse coding agent sends at inference time:

markdown
You are a CUDA sparse kernel specialist.

Chat template is applied with enable_thinking=False, matching the way the Dr.Sparse eval pipeline drives the model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

DiogenesChen122

Model Tree

Base

Qwen/Qwen3.8-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Adapter

Format: PEFT LoRA (~934 MB)
Rank / α: 64 / 128
Target modules (12):
markdown
```
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
```
(dense + MLP) and
markdown
```
in_proj_qkv, in_proj_z, in_proj_a, in_proj_b, out_proj
```
(linear-attention layers of Qwen3.8's hybrid stack). Vision tower is excluded — the adapter lands only on model.language_model.*.
base_model_name_or_path: Qwen/Qwen3.8-27B.

Data

Teacher: GPT-5.6, tree-search runs across the entire Dr.Sparse evaluation corpus (level 1–4, multiple GPU generations).
Selection: rejection-sampling SFT — keep a turn iff the kernel compiled, verified numerically, and beat cuSPARSE by ≥ 1.05×. 4046 kept (3841 train / 205 val, held out by run_id) out of 5420 mined turns.
Op mix (4046 samples): 1023 spmm_k8 · 810 spmm_k32 · 435 spmm_k128 · 329 spmm_k256 · 753 spgemm · 696 spmv. Much better balance than the earlier 1.5× threshold — SpMV and SpGEMM are meaningfully represented now.
Corpus size: ~57 M tokens/epoch, med sequence 13.5 k tokens, max 22 k.

Training

Table

framework	`verl` 0.8.0, SFT trainer
precision	bfloat16 (frozen base + bf16 compute; LoRA in fp32)
optimizer	AdamW, LR 1e-5, cosine, 3 % warmup, weight decay 0
epochs / steps	3 / 2880 total optimizer steps
global batch	4 (dynamic sequence packing)
max_length	24 576 (samples above this dropped at build time)
loss mask	assistant-turn only (pre-flight asserted)

Val loss (best) = 0.02852 at the end of epoch 3. Training curves in the Dr.Sparse W&B project drsparse_distill.

Loading

python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3.8-27B"
lora_id = "DiogenesChen122/Qwen3.8-27B-Lora-20260826"

tok = AutoTokenizer.from_pretrained(lora_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16,
    device_map="auto", trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, lora_id)

Merge and serve via vLLM:

bash
CUDA_VISIBLE_DEVICES=0,1 vllm serve <merged-dir> \
  --served-model-name Qwen/Qwen3.8-27B \
  --tensor-parallel-size 2 --dtype bfloat16 \
  --language-model-only --trust-remote-code \
  --max-model-len 32768 --port 8028

Coder system prompt

The adapter was trained with the exact system turn the Dr.Sparse coding agent sends at inference time:

markdown
You are a CUDA sparse kernel specialist.

Chat template is applied with enable_thinking=False, matching the way the Dr.Sparse eval pipeline drives the model.