Ionio-ai

Qwen3.5-0.8B-Ecommerce-Extraction-LoRA

README

License: apache-2.0

Required prompt format

Training and evaluation used the base model's chat template with exactly two messages. The dataset meta_prompt was ignored.

System message:

text
You extract structured filters from e-commerce search queries.
Return only one valid JSON object, with no markdown or explanation.
Your output must validate against the supplied JSON Schema: include every required key, preserve nesting, do not add keys, and keep arrays as arrays.
Fill values only when stated or clearly implied by the query. Use JSON null for a required scalar field whose value is not available in the query.
Preserve the exact spelling and capitalization of every JSON key.

User message:

text
E-commerce query:
{query}

JSON Schema:
{compact_json_schema}

The schema must be value-free. Every object key is required, extra keys are forbidden, and scalar types allow JSON null. If a required scalar is unavailable, output JSON null—not None, an omitted key, or the string "null". Evaluation used temperature 0, top_p=1, a 4096-token output allowance, and reasoning disabled (enable_thinking=False).

End-to-end example

User query:

text
men's Nike running shoes in red under $100

JSON Schema supplied in the user message:

json
{"type":"object","properties":{"product_type":{"type":["string","null"]},"brand":{"type":["string","null"]},"color":{"type":["string","null"]},"price_max":{"type":["number","null"]}},"required":["product_type","brand","color","price_max"],"additionalProperties":false}

Expected assistant output:

json
{"product_type":"running shoes","brand":"Nike","color":"red","price_max":100}

Loading with Transformers and PEFT

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3.5-0.8B"
adapter_id = "Ionio-ai/Qwen3.5-0.8B-Ecommerce-Extraction-LoRA"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto", torch_dtype="auto")
model = PeftModel.from_pretrained(base, adapter_id)

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": f"E-commerce query:\n{query}\n\nJSON Schema:\n{compact_schema}"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt",
    enable_thinking=False,
).to(model.device)
output = model.generate(inputs, do_sample=False, max_new_tokens=4096)
print(tokenizer.decode(output[0, inputs.shape[-1]:], skip_special_tokens=True))

Define SYSTEM_PROMPT exactly as shown above. Validate the returned JSON against the supplied schema before using it.

Training

TRL SFTTrainer 0.29.1, assistant-only loss
9,341 training and 549 validation examples
LoRA rank 32, alpha 64, dropout 0.05
Target modules: attention projections, MLP projections, and Qwen3.5 GDN projections (in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj)
2 epochs, cosine schedule, peak learning rate 2e-04
Effective batch size 128; maximum sequence length 2048
BF16 training, gradient checkpointing, Liger kernel, grouped-by-length batches
Trainable parameters: 21,645,312
Training time: 29.5 minutes on one NVIDIA RTX PRO 6000 Blackwell Workstation Edition

Held-out evaluation

The following results use the complete 1,095-example held-out test set and greedy vLLM inference. Qwen3.5 was evaluated after merging because vLLM 0.27.1's dynamic LoRA path did not correctly apply all GDN adapter projections; the merged weights are mathematically equivalent to applying this adapter in PEFT/Transformers.

Table with columns: Metric, Result
Metric	Result
Strict JSON	99.91%
Schema valid	99.73%
Exact match	30.32%
Case-insensitive exact	34.61%
Leaf precision	88.59%
Leaf recall	88.50%
Leaf F1	88.52%
Key F1	99.48%
Aligned type accuracy	99.91%

Strict JSON requires the entire response to parse without fences, commentary, or repair. Schema validity checks required keys, types, nesting, arrays, and extra keys. Exact match is case-sensitive and all-or-nothing. Leaf F1 is the macro per-example F1 over flattened (JSON path, typed value) pairs; Key F1 ignores values. Null accuracy measures correct null output on gold-null paths, with no-null examples defined as 100%. No “recoverable” JSON is credited as strict JSON.

Machine-readable training and evaluation records and the PDF report are included in this repository.

Limitations

Performance is measured on the source dataset's held-out split and depends on its annotations and normalization conventions. Other languages, schemas, prompts, base-model revisions, inference engines, or sampling settings may differ. Exact spelling and capitalization matter. Always validate output and do not treat inferred attributes as verified product facts.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Ionio-ai

Model Tree

Base

Qwen/Qwen3.5-0.8B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality