GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
diegogs1451
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
diegogs1451
Model Tree
Qwen/Qwen2.5-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
The model takes Spanish administrative/legal texts from official journals and produces concise, formal titles ready for publication. It is specialized in the three official journals that make up the dUO acronym:
DOUE — Diario Oficial de la Unión Europea (Official Journal of the European Union)
BOE — Boletín Oficial del Estado (Spanish Official State Bulletin)
BOPA — Boletín Oficial del Principado de Asturias (Official Bulletin of Asturias)
Developed by: Diego Gonzalez Suarez
Shared by: diegogs1451
Funded by: University of Oviedo
Model type: Qwen2ForCausalLM (decoder-only causal language model with LoRA adapters)
Language(s) (NLP): Spanish
License: Apache 2.0
Finetuned from model: Qwen/Qwen2.5-7B-Instruct
The model is designed to generate summarization titles for administrative and legal texts from official journals (BOE, DOUE, BOPA). Given a Spanish administrative/legal text as input, it outputs a concise, clear, formal title preserving the essential administrative meaning while eliminating unnecessary details and redundancies.
This model is specialized for Spanish administrative text summarization and may not perform well on general text summarization tasks, other languages, or informal content.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_name = "Qwen/Qwen2.5-7B-Instruct"adapter_path = "diegogs1451/qwen2.5-7B-Instruct-dUO-finetuned-20260702-3epochs" tokenizer = AutoTokenizer.from_pretrained(base_model_name)model = AutoModelForCausalLM.from_pretrained( base_model_name, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, adapter_path) system_prompt = ( "Eres un asistente experto en resumir textos oficiales administrativos y jurídicos " "del DOUE (Diario Oficial de la Unión Europea), del BOE (Boletín Oficial del Estado) " "y del BOPA (Boletín Oficial del Principado de Asturias) en títulos breves, claros, " "formales y listos para publicación. Conserva el significado administrativo esencial " "eliminando detalles innecesarios y redundancias.") def generate_title(text): messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": text}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt" ).to(model.device) outputs = model.generate(inputs, max_new_tokens=128, temperature=0.1) return tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)Fine-tuned on a custom dataset of approximately 200 examples (~86k tokens) sourced from:
The dataset field used for training is text, paired with a system prompt instructing the model to summarize administrative texts into formal titles.
| Parameter | Value |
|---|---|
| LoRA rank (r) | 8 |
| LoRA alpha | 16 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Per device batch size | 2 |
| Gradient accumulation steps | 2 (effective batch size: 4) |
| Max steps | 300 |
| Num epochs | 3 |
A held-out portion of the DOUE/BOE/BOPA dataset was used for evaluation.
| Step | Epoch | Train Loss | Train Token Acc. | Eval Loss | Eval Token Acc. | Learning Rate |
|---|---|---|---|---|---|---|
| 100 | 1 | 0.1936 | 94.8% | 0.1651 | 96.6% | 1.34e-5 |
| 200 | 2 | 0.1286 | 95.6% | 0.1502 |
The model achieves eval loss of 0.1456 and token accuracy of 96.6% on the held-out set after 3 epochs of fine-tuning, with consistent improvement in training metrics and stable evaluation performance across checkpoints.
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_name = "Qwen/Qwen2.5-7B-Instruct"adapter_path = "diegogs1451/qwen2.5-7B-Instruct-dUO-finetuned-20260702-3epochs" tokenizer = AutoTokenizer.from_pretrained(base_model_name)model = AutoModelForCausalLM.from_pretrained( base_model_name, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, adapter_path) system_prompt = ( "Eres un asistente experto en resumir textos oficiales administrativos y jurídicos " "del DOUE (Diario Oficial de la Unión Europea), del BOE (Boletín Oficial del Estado) " "y del BOPA (Boletín Oficial del Principado de Asturias) en títulos breves, claros, " "formales y listos para publicación. Conserva el significado administrativo esencial " "eliminando detalles innecesarios y redundancias.") def generate_title(text): messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": text}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt" ).to(model.device) outputs = model.generate(inputs, max_new_tokens=128, temperature=0.1) return tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)| Learning rate | 2e-5 |
| LR scheduler | Linear with 10-step warmup |
| Optimizer | AdamW (betas: 0.9, 0.999; eps: 1e-8) |
| Max gradient norm | 1.0 |
| Max sequence length | 512 tokens |
| Precision | bfloat16 mixed precision |
| Seed | 42 |
| Training regime | bf16 mixed precision |
| 6.73e-6 |
| 300 | 3 | 0.0965 | 97.0% | 0.1456 | 96.6% | 6.67e-8 |