GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
diegogs1451
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
diegogs1451
Model Tree
Qwen/Qwen2.5-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
The model takes Spanish administrative/legal texts from official journals and produces concise, formal titles ready for publication. It is specialized in the three official journals that make up the dUO acronym:
DOUE — Diario Oficial de la Unión Europea (Official Journal of the European Union)
BOE — Boletín Oficial del Estado (Spanish Official State Bulletin)
BOPA — Boletín Oficial del Principado de Asturias (Official Bulletin of Asturias)
Developed by: Diego Gonzalez Suarez
Shared by: diegogs1451
Funded by: University of Oviedo
Model type: Qwen2ForCausalLM (decoder-only causal language model with LoRA adapters)
Language(s) (NLP): Spanish
License: MIT
Finetuned from model: Qwen/Qwen2.5-7B-Instruct
The model is designed to generate summarization titles for administrative and legal texts from official journals (BOE, DOUE, BOPA). Given a Spanish administrative/legal text as input, it outputs a concise, clear, formal title preserving the essential administrative meaning while eliminating unnecessary details and redundancies.
This model is specialized for Spanish administrative text summarization and may not perform well on general text summarization tasks, other languages, or informal content.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelbase_model_name = "Qwen/Qwen2.5-7B-Instruct"adapter_path = "diegogs1451/qwen2.5-7B-Instruct-dUO-finetuned-20260706-3epochs"tokenizer = AutoTokenizer.from_pretrained(base_model_name)model = AutoModelForCausalLM.from_pretrained(base_model_name,torch_dtype=torch.bfloat16,device_map="auto")model = PeftModel.from_pretrained(model, adapter_path)system_prompt = ("Eres un asistente experto en resumir textos oficiales administrativos y jurídicos ""del DOUE (Diario Oficial de la Unión Europea), del BOE (Boletín Oficial del Estado) ""y del BOPA (Boletín Oficial del Principado de Asturias) en títulos breves, claros, ""formales y listos para publicación. Conserva el significado administrativo esencial ""eliminando detalles innecesarios y redundancias.")def generate_title(text):messages = [{"role": "system", "content": system_prompt},{"role": "user", "content": text},]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)outputs = model.generate(inputs, max_new_tokens=128, temperature=0.1)return tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)
Fine-tuned on a custom dataset of approximately 200 examples (~86k tokens) sourced from:
The dataset field used for training is text, paired with a system prompt instructing the model to summarize administrative texts into formal titles.
| Parameter | Value |
|---|---|
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Per device batch size | 2 |
| Max steps | 174 |
| Num epochs | 3 |
| Learning rate | 2e-5 |
A held-out portion of the DOUE/BOE/BOPA dataset was used for evaluation.
| Step | Epoch | Train Loss | Train Token Acc. | Eval Loss | Eval Token Acc. | Learning Rate |
|---|---|---|---|---|---|---|
| 58 | 1 | 0.1538 | 96.4% | 0.1182 | 96.4% | 1.44e-5 |
| 116 | 2 | 0.1173 | 96.0% | 0.1152 |
The model achieves eval loss of 0.1108 and token accuracy of 96.8% on the held-out set after 3 epochs of fine-tuning, with consistent improvement in training metrics and stable evaluation performance across checkpoints.
| LR scheduler | Linear |
| Optimizer | AdamW |
| Max sequence length | 512 tokens |
| Precision | bfloat16 mixed precision |
| Training regime | bf16 mixed precision |
| 7.47e-6 |
| 174 | 3 | 0.0812 | 97.8% | 0.1108 | 96.8% | 5.75e-7 |