Details
- Base: Qwen/Qwen3-1.7B (Apache-2.0)
- Method: QLoRA (PEFT LoRA + TRL SFTTrainer, vanilla
transformers)
- Task: single-file git diff → one Conventional Commits subject line
- Trained on: marzoukbaig14/committed-train (~58k filtered CommitChronicle commits, 16 languages)
Usage
The trained behavior depends on the exact prompt rendering used in training (a canonical zero-shot Diff:\n{diff} format with enable_thinking=False) plus the GBNF grammar applied at decode time. Loading the adapter with a bare prompt will not reproduce the evaluated output. To match what was evaluated, run it through the project's engine.py, or use the FastAPI / Gradio Space, or the CLI (git diff | committed --model 1.7b). See github.com/marzoukbaig14/Committed.
To load the adapter on top of the base for your own use:
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B")
model = PeftModel.from_pretrained(base, "marzoukbaig14/committed-qwen3-1.7b-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")
Results
This 1.7B fine-tune is the stronger of the two sizes overall, with its main edge in specificity (0.67 vs the 0.6B's 0.55); the margins on the other axes are small (graded 2.14 vs 2.09). Both base models feat-collapse (~87–96% feat); fine-tuning breaks it on both. The full four-arm comparison (0.6B/1.7B × base/fine-tune, all DeepSeek-judged — not comparable to any earlier Gemini figures), the feat-collapse analysis, and the judge validation are in the merged-model card and the eval writeup:
License
Apache-2.0, inherited from the Qwen3-1.7B base.
Citation
Trained with TRL. Dataset derived from CommitChronicle (Eliseeva et al., From Commit Message Generation to History-Aware Commit Message Generation, arXiv:2308.07655).