Details
- Base: Qwen/Qwen3-0.6B (Apache-2.0)
- Method: QLoRA (PEFT LoRA + TRL SFTTrainer, vanilla
transformers)
- Task: single-file git diff → one Conventional Commits subject line
- Trained on: marzoukbaig14/committed-train (~58k filtered CommitChronicle commits, 16 languages)
Usage
The trained behavior depends on the exact prompt rendering used in training (a canonical zero-shot Diff:\n{diff} format with enable_thinking=False) plus the GBNF grammar applied at decode time. Loading the adapter with a bare prompt will not reproduce the evaluated output. To match what was evaluated, run it through the project's engine.py, or use the FastAPI / Gradio Space, or the CLI — see github.com/marzoukbaig14/Committed.
To load the adapter on top of the base for your own use:
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B")
model = PeftModel.from_pretrained(base, "marzoukbaig14/committed-qwen3-0.6b-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
Results
The 0.6B fine-tune matches the 1.7B on type-correctness and faithfulness at a third the size, with the gap concentrated in specificity. Both base models feat-collapse (~87–96% feat); fine-tuning breaks it on both. The full four-arm comparison (0.6B/1.7B × base/fine-tune, all DeepSeek-judged), the feat-collapse analysis, and the judge validation are in the merged-model card and the eval writeup:
License
Apache-2.0, inherited from the Qwen3-0.6B base.
Citation
Trained with TRL. Dataset derived from CommitChronicle (Eliseeva et al., From Commit Message Generation to History-Aware Commit Message Generation, arXiv:2308.07655).