liangzhidanta

Qwen3-8B-CC-SFT-v2-2e

README

License: apache-2.0

Key Result

canonical303 Pass@1: 54.79% (166/303) — 4090 run at T=0.4 (temperature selected on an independent dev set; see Temperature Selection). Wilson 95% CI: [49.16%, 60.29%]. 303/303 valid environments, 0 infrastructure failures.

Table with columns: Model, Training Stage, Solved / 303, Pass@1
Model	Training Stage	Solved / 303	Pass@1
Qwen3-8B	Base	10/303	3.30%
Qwen3-8B-CC-SFT-v1	General Agent SFT	106/303	34.98%
Qwen3-8B-CC-SFT-v2	Compact-aware SFT (1 epoch)	144/303	47.52%
Qwen3-8B-CC-SFT-v2-2e	Compact-aware SFT (2 epochs)	166/303	54.79%

vs the published v2 (1-epoch) checkpoint, paired on identical 303 tasks (same 4090 harness family):

+22 solved tasks (+7.27 pp)
McNemar exact test p = 0.033
95% paired bootstrap CI = [1.0, 13.5] pp

Table with columns: Paired outcome (v2-2e vs v2), Tasks
Paired outcome (v2-2e vs v2)	Tasks
Both solved	106
v2-2e only	60
v2 only	38
Neither	99

Temperature sweep on canonical303 (this checkpoint, one run per temperature): T=0.4 → 166/303 (54.79%), T=0.5 → 159/303 (52.48%, paired p = 0.53 vs T=0.4; 303/303 completed, 2 environment failures counted as unsolved). T=0.4 is the best observed temperature for this checkpoint.

For reference, the v2-1e checkpoint at T=0.5 solved 159/303 (52.48%, paired p = 0.18 vs its T=0.3 run) — v2-2e @ T=0.4 and v2-1e @ T=0.5 are not statistically separable (p = 0.55), but v2-2e @ T=0.4 is the only configuration with a significant paired gain over the published v2 result.

Training Dataset and Recipe

Dataset: liangzhidanta/claude-code-glm53-swesmith-compact-trajectories

This checkpoint is the second epoch of the COMPACT_FOCUSED recipe over the deployment-aligned (≤40960-token) subset of that dataset, replayed together with the v1 trajectory set:

Table with columns: Recipe component, Count, Description
Recipe component	Count	Description
V1 replay	1003	Execution-verified v1 agent trajectories (from claude-code-glm53-swesmith-trajectories)
Compact-summary segments	239	Real Claude Code native compact request → GLM-5.3 state summary
Post-compaction continuation segments	387	Real model-visible post-compaction context → agent continuation
Total (COMPACT_FOCUSED ≤40K)	1629	All segments ≤40960 tokens

Dataset-level provenance: 189 source trajectories (138 one-compact, 51 two-compact) → 240 native compact events → 890 context-correct segments (240 compact-summary + 403 post-compaction + 247 optional pre-compact); the training subset is the ≤40K deployment-aligned split (626 segments) plus V1 replay. Segments are split at real compaction boundaries with prefixes rebuilt from the actual model-visible wire request (context-correct format — see the v2 model card for details).

Epoch 1 (produces Qwen3-8B-CC-SFT-v2): initialized from v1, 367 optimizer steps.
Epoch 2 (this checkpoint): initialized from v2-1e, same 1629-sample recipe in reshuffled order, fresh optimizer and fresh cosine schedule, 367 optimizer steps.

Training Details

Table with columns: Value
	Value
Initial checkpoint	Qwen3-8B-CC-SFT-v2 (1-epoch compact-aware SFT)
Lineage	Qwen3-8B → SFT-v1 → SFT-v2 (epoch 1) → SFT-v2-2e (epoch 2)
Framework	slime v0.3.2 (Megatron-LM backend)
Fine-tuning type	Full-parameter continued SFT
Epoch	2nd epoch over the COMPACT_FOCUSED recipe (1629 samples)
Optimizer	Adam (β₁=0.9, β₂=0.95), weight decay 0.1
Learning rate	5e-6, cosine decay to 5e-7 (fresh schedule per epoch)
Warmup	0.1
Context	40960 (deployment REAL40K; silent truncation disabled)
Steps	367 optimizer steps (734 cumulative across both epochs)
Batch size	rollout batch 4 / global batch 4
Hardware	8× RTX 4090 24GB
Parallelism	TP=8, DP=1, PP=1, CP=1, sequence parallel

Losses are reported as training information only, not as capability metrics.

Training curve (both epochs, 734 optimizer steps)

TensorBoard view (masked SFT loss and per-batch rollout response lengths over both epochs):

Temperature Selection

T=0.4 (jointly with the checkpoint choice) was selected on an independent 60-task development set (task-id and repo disjoint from canonical303), then evaluated once on canonical303 — no post-hoc selection on this benchmark. On the development set, all checkpoint/temperature pairs were statistically indistinguishable (paired McNemar p > 0.3); the canonical303 run confirms the selected configuration with a significant paired gain over the published v2 result (p = 0.033).

Evaluation Protocol

Table with columns: Item, Value
Item	Value
Task set	canonical303 (303 tasks)
Task source	SWE-smith-derived, repo-disjoint from training repositories
Environment	303/303 valid Docker environments
Context	40960 tokens (CC-visible context = SGLang real context)
Context management	Claude Code native automatic context compaction (ON)
Sampling	temperature = 0.4, top_p = 0.95
Max agent calls	32
Metric	Pass@1, 1 rollout per task
Harness	Claude Code 2.1.258 compatible harness
Serving	SGLang 0.5.15.post1 (inference TP=2; independent of training parallelism)

This is an internal SWE-smith-derived evaluation. It is not SWE-bench Verified and should not be interpreted as a leaderboard score. SWE-bench Verified results are not reported in this release.

Model Lineage

text

Qwen/Qwen3-8B        ↓execution-verified GLM-5.3 trajectories        ↓Qwen3-8B-CC-SFT-v1          (general agent SFT)        ↓compact-aware targeted supervision, epoch 1        ↓Qwen3-8B-CC-SFT-v2          (144/303, T=0.3)        ↓same recipe, epoch 2        ↓Qwen3-8B-CC-SFT-v2-2e       (166/303, T=0.4)        ↓verifiable GRPO (planned)

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM model_id = "liangzhidanta/Qwen3-8B-CC-SFT-v2-2e"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

This checkpoint uses the original Qwen3 tokenizer and chat template. For coding-agent evaluation, the model was served through SGLang with a Qwen3 reasoning parser and tool-call parser, behind a Claude-Code-compatible harness with native context compaction enabled (T=0.4, top_p=0.95).

Limitations

The second epoch reuses the same 1629-sample recipe — no new supervision data was added.
Compact summaries are generated by GLM-5.3 and may contain residual errors.
Task pool shared with the v1/v2 lineage (same-task, different-trajectory supervision).
The internal 303-task benchmark is not SWE-bench Verified; SWE-bench Verified results are not reported in this release.
Performance may depend on harness/tool configuration; tool-use performance should not be interpreted as standalone raw-model benchmark performance.
Evaluated on canonical303 at T=0.4 and T=0.5 (single rollout per temperature); other temperatures are untested.

Citation

bibtex

@model{qwen3_8b_cc_sft_v2_2e,  title={Qwen3-8B-CC-SFT-v2-2e: Compact-Aware Coding Agent},  author={liangzhidanta},  year={2026},  url={https://huggingface.co/liangzhidanta/Qwen3-8B-CC-SFT-v2-2e}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

liangzhidanta

Model Tree

Base

liangzhidanta/Qwen3-8B-CC-SFT-v2

Fine-tuned

this model

Input Modalities

Text

Output Modalities