Training
- Base:
Qwen/Qwen3.5-4B (frozen)
- LoRA rank 16, α = 32, dropout 0.05; AdamW, lr 1e-4, 3 epochs, single seed
- 681 training episodes for which Kimi-K3 produced a usable correct answer and a
consistent explanation; target = GPT-6 explanation followed by the three labels
- Prompt:
experiments/qwen35_cot_distill/prepared/prompt.txt; output budget 1,536 tokens
Results (200-episode test set)
Table with columns: Joint acc. | Joint acc. |
|---|
| Untuned Qwen3.5-4B, same explanation prompt | 50.5 |
| This adapter | 82.5 (long slice 62.5) |
Label-only adapter (avinashsystems/termshift-qwen3.5-4b-lora) | 87.0 |
Explanation supervision improves over the untuned baseline but does not exceed
label-only adaptation; the comparison differs in data, prompt, target length and
output budget.
Usage
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen3.5-4B"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(model, "avinashsystems/termshift-qwen3.5-4b-lora-explain-gpt6")
Limitations
Trained and evaluated on synthetic episodes only; performance on independently
drafted terms is not established. Research artifact, not legal advice.