Training
- Base:
llmfan46/Qwen3.6-35B-A3B-uncensored-heretic, revision dbfd9eb0cdc7c33fc970b06429f6e043b6851190.
- Source code and dataset revision:
41bc78a3a7be8b1bfdc347dc3b9dbe56f68f5a33.
- Data:
data/hot-api-v2/dataset_dialog_train.json (750 records) and dataset_dialog_test.json (28 held-out records).
- One epoch, 188 optimizer steps; batch size 4, gradient accumulation 1, one GPU; effective batch size 4.
- LoRA rank 16, alpha 32, dropout 0; attention and expert MLP projections, plus the two tool-context token rows.
- Learning rate 0.0002, weight decay 0.01, AdamW 8-bit optimizer, linear scheduler, seed 3407.
- 4-bit base loading, Unsloth gradient checkpointing, maximum sequence length 12,000 tokens.
- Average training loss: 1.4119867223. Trainer runtime: 1,826.73 seconds, including held-out loss evaluations.
- Final held-out loss: 1.3846312761 on all 28 test records at step 188. Held-out loss was evaluated every 10 steps and at the end.
- The final adapter is the last training step; no best-checkpoint selection was performed.
- Serving weights SHA-256:
0f75651c225771b58d4d22e0173ae8b4d1c7c48e7a9fe0d3d42d67926de2596b.
Serving
Use the pinned base checkpoint with this dialog LoRA, is_3d_lora_weight: true, the adapter tokenizer and its chat template. The export contains 256 experts, 80 converted expert tensors, and embedding LoRA for tool-context token IDs 248077 and 248078. The repository includes the embedding compatibility and MoE conversion helpers.
The tokenizer and chat template are byte-identical to the preceding dialog release. The adapter rank is now 16; the serving maximum LoRA rank must be at least 16. This adapter is separate from the specialized routing and extraction adapter.
Evaluation limits
Stage 2 was not run. Generated-answer evaluation and live backend replay have not been run for this release. Held-out loss does not establish response quality or an improvement over earlier releases.
Datasets, raw conversations, optimizer states, training logs, and evaluation outputs are excluded from this repository.