What it does
Trained on 56,253 instruction samples: 70% ECInstruct (generic e-commerce) + 30% synthetic Magento-schema data generated from the Magento Luma sample catalog (products fully disjoint between train and eval). Four task shapes:
Attribute extraction — product text (or a raw Magento custom_attributes payload) → JSON:
target attribute: size
product title: Puma Suede green sneakers size 43
→ [{"attribute": "size", "value": "43"}]
Absent attributes are reported as "None" rather than hallucinated.
Product QA — a question answered strictly from given product data.
Relevance classification — query + product → graded relevance option (ESCI-style A–D).
Relevance ranking — query + lettered product list → ranked letters (B,A,C).
Usage — adapter (unsloth / peft)
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
"gtstadeu/qwen3.5-4b-ec-magento", max_seq_length=2048, load_in_4bit=True)
FastLanguageModel.for_inference(model)
messages = [{"role": "user", "content":
"Extract the value of the target attribute from the given product information "
"and output it as JSON. If the attribute is not present, output None as the value.\n\n"
"target attribute: size\nproduct title: Puma Suede green sneakers size 43"}]
text = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, enable_thinking=False, tokenize=False)
inputs = tokenizer(text=text, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Plain peft also works: PeftModel.from_pretrained(base_model, "gtstadeu/qwen3.5-4b-ec-magento") over Qwen/Qwen3.5-4B.
Usage — GGUF / Ollama
hf download gtstadeu/qwen3.5-4b-ec-magento --include 'gguf/*' --local-dir .
cd gguf && ollama create qwen3.5-4b-ec-magento -f Modelfile
ollama run qwen3.5-4b-ec-magento --think=false "target attribute: color ..."
Ollama ≥ 0.31 uses its built-in Qwen3.5 renderer and defaults to thinking mode — pass --think=false (CLI) or "think": false (API) for the trained fast-path output. Requires an Ollama version with qwen35 architecture support. ~95 tok/s on an RTX 3070.
Training recipe
Table with columns: Parameter, Value| Parameter | Value |
|---|
| Method | QLoRA (4-bit NF4 base, bf16 compute) via Unsloth |
| LoRA | r=8, alpha=16, targets q/k/v/o/gate/up/down_proj |
| Trainable params | 10.6M of 4,550M (0.23%) |
| Batch | 1 × grad_accum 16 (effective 16), max_seq_length 2048 |
| Optimizer / LR | paged_adamw_8bit, 2e-4 cosine, 1 epoch, seed 42 |
| Data | 56,253 samples: 39,377 ECInstruct + 16,876 Magento-synthetic |
Evaluation
Greedy decoding, identical prompts and chat template across all models; base model evaluated zero-shot with the same harness. ec50k is an intermediate adapter trained on ECInstruct only (not published) — shown to isolate what the Magento data adds.
Magento held-out set (2,969 samples, 475 products never seen in training):
Table with columns: Task · metric, Base, ec50k, this model| Task · metric | Base | ec50k | this model |
|---|
| Attribute extraction · F1 | 0.015 | 0.320 | 0.938 |
| Attribute extraction · parse failures | 71.5% | 8.1% | 0% |
| Product QA · token-F1 | 0.041 | 0.299 | 0.943 |
| Relevance classification · accuracy | 0.194 | 0.770 |
ECInstruct held-out set (2,000 samples):
Table with columns: Task · metric, Base, ec50k, this model| Task · metric | Base | ec50k | this model |
|---|
| Attribute extraction · F1 | 0.000 | 0.685 | 0.646 |
| Query→product rank · top-1 | 0.013 | 0.670 | 0.650 |
| Relevance classification · accuracy | 0.258 | 0.678 | 0.685 |
| Answerability · accuracy | 0.503 | 0.778 | |
Limitations — read before relying on the numbers
- The Magento eval is synthetic-on-synthetic. Eval tasks were generated with the same templates as training data (products fully disjoint). It validly measures schema adherence — JSON format, Magento attribute vocabularies, the None-when-absent rule — but overstates production quality on real catalogs and real user queries.
- The base model's near-zero extraction scores are dominated by format non-adherence (it answers in prose); they understate its underlying capability, though prose output is itself a blocker for programmatic use.
- English only. Inputs should fit a 2,048-token budget (strip HTML from product descriptions).
- Free-form generation is weak (trained r=8, extraction-focused); use it for structured tasks, not copywriting.
- Use greedy decoding (
do_sample=False / temperature 0) — that's how it was evaluated.
Provenance
Table | |
|---|
| Training run | qwen3.5-4b-r8-mix56k-e1 (2026-07-03) |
| Adapter sha256 | 8e7a7f89da314bd7c206b1f2fad2ad9eaaa4b0e988f72ea8f4592aa099bc6b42 |
| Train set sha256 | afb7e664cda4490597c1914db6aff94933991a56d2175435666f8f6b7a726532 (mixture_train.jsonl, 56,253 rows) |
| Eval set sha256 | 8feafdb2… (magento_eval.jsonl) · 2583a61e… (ecinstruct_eval.jsonl) |
| Stack | torch 2.6.0+cu124 · transformers 5.5.0 · unsloth 2026.6.9 |
References