Results
Table with columns: metric, before, after| metric | before | after |
|---|
| holdout NLL (strands corpus) | 2.172 | 1.581 (Δ −0.59, −27%) |
Training: 600 steps, LoRA r=32 (α=64, dropout 0.05) on q/k/v/o + gate/up/down,
packed 1024-token blocks, bs 4 × accum 2, cosine LR 1e-4, bf16, single L40S.
Use with SLM (self-learning provider for Strands Agents)
from slm import SLM
from strands import Agent
model = SLM("cagataydev/strands-qwen3-0.6b")
agent = Agent(model=model)
agent("How do I create a custom tool in Strands?")
model.save("experience.pt")
Or plain transformers/peft:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "cagataydev/strands-qwen3-0.6b").merge_and_unload()
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
SLM model family
Table with columns: repo, base, params, holdout NLL Δ| repo | base | params | holdout NLL Δ |
|---|
| cagataydev/strands-qwen3-vl-2b | Qwen3-VL-2B-Instruct | 2B | 1.85 → ~1.0 |
| cagataydev/strands-gemma4-e2b | Gemma 4 E2B (QAT mobile) | 2B eff. | 2.69 → 1.26 |
| cagataydev/strands-qwen3-0.6b | Qwen3-0.6B | 0.6B | 2.17 → 1.58 |