Results
Validation loss fell at every checkpoint, and the best checkpoint is the final one -- the model was still improving when the epoch ended, with no sign of overfitting: 0.182 -> 0.128 over 9 checkpoints, with token accuracy rising to 96.5%. Measured on the full held-out validation split at every checkpoint.
Table with columns: Step, Validation loss, Token accuracy| Step | Validation loss | Token accuracy |
|---|
| 100 | 0.18177 | 95.48% |
| 200 | 0.15792 | 95.93% |
| 300 | 0.14828 | 96.15% |
| 400 | 0.14368 | 96.19% |
| 500 | 0.13787 | 96.31% |
| 600 | 0.13343 | 96.40% |
| 700 | 0.12974 | 96.50% |
| 800 | 0.12855 | 96.51% |
| 887 | 0.12835 | 96.52% |
What it is
A QLoRA fine-tune of microsoft/Phi-4-mini-reasoning,
trained on HIT's product feed and knowledge base. The base model is open-weights,
American-built, and reasons before it answers; the adapter teaches it this catalog
and HIT's order policy, not general promotional-products trivia.
Intended use
Internal support for HIT sales reps and distributors, with catalog records
supplied in the prompt. The pipeline retrieves the relevant product rows and
policy chunks and puts them in context; the model reasons over them. It is not
a catalog lookup on its own, and it is not a substitute for the live feed.
Out of scope
- Setting your sell price. The model quotes NET/EQP off the catalog and says
so, but your margin and freight are not in the data, so the last step of a
customer quote is yours.
- Answering without context. Asked about a SKU with no record in the prompt,
the model is trained to say it does not have one. That is the designed
behaviour, and it is more trustworthy than a confident answer from memory.
Training data
15,779 supervised examples generated from HIT's product feed and knowledge base.
Table with columns: Split, Examples, Source groups, Mean answer chars| Split | Examples | Source groups | Mean answer chars |
|---|
| train | 14,201 | 3,570 | 607.9 |
| val | 790 | 188 | 620.9 |
| test | 788 | 208 | 581.0 |
Table with columns: Task, Examples, What it teaches| Task | Examples | What it teaches |
|---|
quantity_pricing | 4,000 | quantity and budget arithmetic across the price breaks |
product_qa | 4,000 | grounded single-field lookups (imprint area, colors, origin, MOQ) |
spec_extraction | 3,999 | free text to a structured spec JSON |
policy_lookup | 1,666 | lead times, minimums, less-than-minimum, compliance, returns |
Splits are grouped by source_id, not sampled per row: every colour variation of a style moves together, so a held-out score cannot be inflated by having seen the same product in training under a different variation code.
Every example was generated only where the answer is derivable from the source
record. Nothing plausible-sounding was filled in: a model trained on invented
specs learns to invent specs.
Training procedure
Table with columns: Setting, Value| Setting | Value |
|---|
| Base model | microsoft/Phi-4-mini-reasoning |
| Method | QLoRA (frozen nf4 base) |
| LoRA rank / alpha | 32 / 64 |
| LoRA dropout | 0.05 |
| Target modules | qkv_proj, o_proj, gate_up_proj, down_proj |
| Epochs | 1 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("raj-hitpromo/hit-promo-reasoner", dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("raj-hitpromo/hit-promo-reasoner")
The system prompt matters -- it carries the NET-pricing and grounding rules the
model was trained against. It is in hit_model/data/templates.py as
SYSTEM_PROMPT, and the serving path applies it for you.
Limitations
- Confirm figures before they reach a customer. The model reasons over the
context it is given and shows its arithmetic, but a fluent chain of thought can
still land on a wrong number. Anything going onto a quote should be checked
against the live feed.
- The catalog moves. Prices, lead times and stock change daily; the weights
are a snapshot. This is why retrieval is part of the design -- context comes
from the live feed, so the model tracks the catalog without being retrained.
Provenance
Built with the pipeline in this project; see MODEL_BUILD.md for the full
method. Card generated 2026-09-12 from the config, dataset
stats, and evaluation report that produced this artifact.