GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
BenarbiaEmir
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
BenarbiaEmir
Model Tree
Qwen/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
| Base zero-shot | Base few-shot | This adapter | |
|---|---|---|---|
| Exact match, 9 fields (95% CI) | 19.5% [14.0%, 25.0%] | 22.5% [17.0%, 28.5%] | 83.5% [78.0%, 88.5%] |
| Schema-valid | 91.5% | 92.5% | 98.5% |
| email_type | 61.0% | 66.5% | 96.5% |
| stage | 71.5% | 77.5% | 93.5% |
| deadline | 69.0% | 71.5% | 97.0% |
| Mean latency (T4) | 8.35 s | 8.73 s | 13.35 s |
Half the test set is written by a different generator model than the training data; on that half the adapter reaches 80% exact match. Baselines get the labeling rules as a written prompt; the adapter uses a one-line instruction. Details, plots and error analysis are in the GitHub README.
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig base = "Qwen/Qwen3-4B-Instruct-2507"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", dtype=torch.float16, quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.float16))model = PeftModel.from_pretrained(model, "BenarbiaEmir/qwen3-4b-recruitment-email-json-qlora") email = open("email.txt").read() # include the From:/Date:/Subject: headersmessages = [{"role": "system", "content": "Extract the recruitment email record as JSON (schema v1)."}, {"role": "user", "content": "Email:\n" + email}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=256, do_sample=False)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))The Date: header is used to resolve relative deadlines ("avant vendredi"). Validate the output
against the schema (data/schema.json in the GitHub repo) before using it.
QLoRA: 4-bit nf4 base (double quantization, fp16 compute), LoRA r=16, alpha=32, dropout 0.05 on all linear layers; 824 training emails, 2 epochs (104 steps, effective batch 16), lr 2e-4 cosine, loss on the JSON completion only, max length 1024 (no example truncated). Checkpoint selected by validation loss (105 emails). One free Colab T4.
screening); see the labeling rules in the GitHub repo.import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig base = "Qwen/Qwen3-4B-Instruct-2507"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", dtype=torch.float16, quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.float16))model = PeftModel.from_pretrained(model, "BenarbiaEmir/qwen3-4b-recruitment-email-json-qlora") email = open("email.txt").read() # include the From:/Date:/Subject: headersmessages = [{"role": "system", "content": "Extract the recruitment email record as JSON (schema v1)."}, {"role": "user", "content": "Email:\n" + email}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=256, do_sample=False)print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))