| Base model | Qwen/Qwen3.5-2B, revision 15852e8c16360a2fea060d615a32b45270f8a8fc, Apache-2.0. Base weights are not included: download them from the official repository |
| Architecture | Qwen3.5 hybrid: 6 full-attention layers and 18 Gated-DeltaNet linear-attention layers, plus a vision encoder (Qwen3_5ForConditionalGeneration) |
| Adapter | LoRA r = 16, α = 32, dropout 0.05, fp32 weights, 15.63 M parameters (300 tensors) |
| Target modules | language-model q_proj, k_proj, v_proj, o_proj (full attention), in_proj_qkv, in_proj_z, out_proj (DeltaNet), gate_proj, up_proj, down_proj (MLP). Vision tower and merger frozen |
| Library | PEFT 0.21.2, transformers 5.22.2, PyTorch 2.12 (Apple MPS, bf16 base) |
| Developed by | Team-15 (CodeMetal Forge hackathon) |
| License | Apache-2.0 (same as the base model) |