Base model
Qwen/Qwen3-1.7B
Training
- 4,000 synthetic training records
- 500 validation records
- Two epochs and 250 optimizer steps
- QLoRA with NF4 double quantization
- LoRA rank 16, alpha 32, dropout 0.05
- Effective batch size 32
- BF16 compute on one RTX 5090
- Training time: 38 minutes 44 seconds
Evaluation
Table with columns: Evaluation set, Model decision macro F1, Model citation exact match, Deterministic decision macro F1, Deterministic citation exact match| Evaluation set | Model decision macro F1 | Model citation exact match | Deterministic decision macro F1 | Deterministic citation exact match |
|---|
| Test-1, standard unseen cases | 0.9944 | 95.8% | 1.0000 | 100.0% |
| Test-2, adversarial dialogue | 0.8729 | 91.8% | 0.8880 | 95.6% |
| Test-3, unseen rule changes | 0.7506 | 53.0% | 0.9462 | 75.4% |
Each test set contains 500 records. Macro F1 gives each of the four decision classes equal weight; citation exact match requires the complete set of failed rule IDs. Model scores are reported before deterministic correction. These metrics do not rule out overfitting: the standard test shares synthetic generation templates with training. The base-model baseline used 100 validation records and the adapter used 500, so they are not a matched estimate of improvement. Full accuracy and per-class metrics remain in the project evaluation report.
Test-2 exposed failures on ambiguous and contradictory answers. Test-3 showed that the model sometimes removed applicant values that a changed policy disallowed instead of preserving those values for deterministic evaluation.
Loading the adapter
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base_id = "Qwen/Qwen3-1.7B"adapter_id = "sabber/snh-qwen3-1.7b-loan-adjudication-lora" tokenizer = AutoTokenizer.from_pretrained(adapter_id)base_model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")model = PeftModel.from_pretrained(base_model, adapter_id)
Generation should be deterministic (do_sample=False) with thinking disabled in the Qwen chat template. The complete prompt contract and evaluation code are in the project repository.
Limitations and intended use
This is a synthetic-data research artifact, not a validated lending model. It has not been tested on real applicants, calibrated for credit risk, audited for fairness, or approved for production lending. It may hallucinate fields when answers are ambiguous and may fail when policy rules change. Human review and deterministic verification are required.