Why
Prompting a base instruct model for resume screening produces unstructured, inconsistent prose -- not something you can pipe into an ATS or scoring pipeline. This adapter makes structured JSON output the model's default behavior, not something you have to coax out with prompt engineering.
Training details
- Base model: Qwen2.5-0.5B-Instruct
- Method: LoRA (r=16, alpha=32, dropout=0.05) targeting q/k/v/o projections
- Trainable params: 2,162,688 / 496,195,456 total (0.44%)
- Dataset: 800 train / 96 eval examples, resume text -> structured JSON verdict
- Training: 3 epochs on a free Colab T4 GPU
Results
Table with columns: Epoch, Training Loss, Validation Loss| Epoch | Training Loss | Validation Loss |
|---|
| 1 | 0.4078 | 0.2784 |
| 2 | 0.1956 | 0.1883 |
| 3 | 0.1713 | 0.1714 |
Validation loss tracked training loss closely with no divergence -- no overfitting.
Before vs after
Prompt: Screen this resume for a Backend Engineer position and return a structured verdict. Resume: Bachelor's in CS, 4 years distributed backend experience, skilled in Python, PostgreSQL, Docker, Kubernetes.
Base model (no fine-tuning): unstructured prose with markdown headers (Summary, Key Skills, Experience) -- not machine-parseable.
Fine-tuned model:
{"role": "Backend Engineer", "ats_score": 56, "verdict": "moderate_match", "matched_skills": ["Python", "PostgreSQL", "Docker", "Kubernetes"], "missing_skills": ["Java", "REST APIs", "Redis"], "years_experience": 4}
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "Kus-hal/resume-screener-lora")
tokenizer = AutoTokenizer.from_pretrained("Kus-hal/resume-screener-lora")
Links
Stack
PEFT/LoRA, Hugging Face Transformers, TRL, PyTorch, Colab T4 GPU