Training
Table | |
|---|
| Rows seen | 6,714 (the whole dataset less the holdout; 5,000 were planned) |
| Sessions | 1 |
| Final training loss | 0.1456 |
| Method | QLoRA, 4-bit NF4, r=16, alpha=32 |
| Hardware | Kaggle T4 (16GB), free tier |
Does it help?
On 200 rows held out of training and never seen by the model, mean token loss fell from 2.3668 to 0.1418 (94.0% lower). The same rows were scored before and after training, so this is a like-for-like comparison rather than a training-loss curve.
Table | |
|---|
| Held-out loss, base model | 2.3668 |
| Held-out loss, this adapter | 0.1418 |
| Loss reduction | 2.2250 (94.0% lower) |
| Held-out rows | 200 |
Trained in 1 session on Kaggle's free
GPU, which stops at twelve hours - longer runs resume from the previous
session's checkpoint.
Use
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
model = PeftModel.from_pretrained(base, "sinhal/qwen3-4b-instruct-2507-gsm8k-reasoning")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
Honest limitations
Trained on one dataset for 1 session on
free-tier hardware. It has not been benchmarked, and no claim is made that
it beats the base model at anything. It exists because the pipeline that
produced it runs unattended, and it is published so the work is inspectable.
Built and published automatically by an agent. The base model and dataset
licences above govern use.