GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Despina
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Despina
Model Tree
Qwen/Qwen2.5-0.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Given a sentence and two marked entities, the model outputs only the relation label that holds between them (one label, no explanation).
This repo is a PEFT LoRA adapter, so load the base model and attach the adapter:
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE = "Qwen/Qwen2.5-0.5B-Instruct"ADAPTER = "Despina/Qwen2.5-0.5B-Instruct-re_gentune-2-shot" tokenizer = AutoTokenizer.from_pretrained(ADAPTER)model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER)model.eval() system_prompt = ( "You are a relation extraction system. Be concise and direct. " "Output ONLY the relation type that holds between the two mentioned entities. " "Do not output any explanation, punctuation, or extra text — only the label.")user_prompt = ( "Sentence: Steve Jobs co-founded Apple in Cupertino.\n" "Entity 1: Steve Jobs\n" "Entity 2: Apple\n" "Relation:") messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": user_prompt},]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt").to(model.device) out = model.generate(inputs, max_new_tokens=16, do_sample=False)print(tokenizer.decode(out[0, inputs.shape[-1]:], skip_special_tokens=True).strip())For best results, match the format the model was trained on: a system prompt asking for the label only, and (optionally) two in-context examples before the query — this is the 2-shot regime. A schema-enumerated variant, where the allowed label set for the target dataset is injected into the system prompt, gives the strongest results in the paper.
| Base model | Qwen/Qwen2.5-0.5B-Instruct |
| Method | QLoRA (4-bit NF4, bf16 compute, double quant) |
| LoRA | r = 32, α = 64, dropout = 0.05; targets: q/k/v/o + gate/up/down proj |
| Training data | Despina/re_gentune (GenTune general-domain mixture), 2-shot prompts |
| Objective | Generate the relation label only |
| Epochs | 2 |
| Learning rate | 1e-4 |
| Effective batch | 4 × 2 grad-accum = 8 |
| Max sequence length | 1024 |
GenTune aggregates seven general-domain RE datasets: TACRED, SemEval-2010 Task 8, CoNLL04, NYT11, GIDS, Re-DocRED, and REBEL.
Scored with positive-class micro-F1 (the no-relation class is excluded from the average). On the general-domain benchmarks the model scores 0.83 general-domain average, versus zero-shot GPT-5.4 (0.69) and Claude Sonnet 4.6 (0.66) under a minimal zero-shot protocol. As the paper stresses, this reflects targeted task adaptation rather than any intrinsic superiority of small models. See the paper for the full 30-configuration matrix, literary-domain results, and the RoBERTa discriminative baseline.
Despina/re_gentuneIf you use this model, please cite:
bibtex
@article{christou2026subbillion, title = {Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs on General and Literary Relation Extraction}, author = {Christou, Despina and Tsoumakas, Grigorios}, journal = {arXiv preprint arXiv:2606.22606}, year = {2026}, url = {https://arxiv.org/abs/2606.22606}}import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE = "Qwen/Qwen2.5-0.5B-Instruct"ADAPTER = "Despina/Qwen2.5-0.5B-Instruct-re_gentune-2-shot" tokenizer = AutoTokenizer.from_pretrained(ADAPTER)model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER)model.eval() system_prompt = ( "You are a relation extraction system. Be concise and direct. " "Output ONLY the relation type that holds between the two mentioned entities. " "Do not output any explanation, punctuation, or extra text — only the label.")user_prompt = ( "Sentence: Steve Jobs co-founded Apple in Cupertino.\n" "Entity 1: Steve Jobs\n" "Entity 2: Apple\n" "Relation:") messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": user_prompt},]inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt").to(model.device) out = model.generate(inputs, max_new_tokens=16, do_sample=False)print(tokenizer.decode(out[0, inputs.shape[-1]:], skip_special_tokens=True).strip())@article{christou2026subbillion, title = {Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs on General and Literary Relation Extraction}, author = {Christou, Despina and Tsoumakas, Grigorios}, journal = {arXiv preprint arXiv:2606.22606}, year = {2026}, url = {https://arxiv.org/abs/2606.22606}}