Table of Contents
Model Description
Table with columns: Property, Value| Property | Value |
|---|
| Base Model | gplsi/Aitana-2B-SI-Instruct |
| Architecture | Transformer decoder-only |
| Parameters | ~2.25B |
| Languages | Valencian, Spanish, English |
| License | Apache 2.0 |
Aitana-2B-SI-Instruct-Aligned extends the Aitana-2B-SI-Instruct instruction-tuned model with Direct Preference Optimization (DPO) alignment. This additional training stage improves the model's ability to generate helpful, high-quality responses that better align with human preferences while maintaining its strong multilingual capabilities.
Alignment Details
The model was aligned using Direct Preference Optimization (DPO) with the following configuration:
Table with columns: Hyperparameter, Value| Hyperparameter | Value |
|---|
| Method | DPO (Direct Preference Optimization) |
| Learning rate | 5e-6 |
| Epochs | 1 |
| Beta | 0.1 |
| LR Scheduler | Linear |
| Total Samples | 146,180 |
| English Samples | 80,308 |
| Spanish Samples | 30,072 |
|
The DPO alignment was performed using curated preference pairs that teach the model to prefer more helpful, accurate, and well-structured responses.
Training Data
The base instruction model was trained on the ALIA Instruction/v12 dataset. This DPO-aligned variant was further aligned using the Alignment/v8 dataset, composed of the following preference data:
The alignment data focused on English, Spanish, and Valencian preference pairs, with the distribution: 80,308 English, 30,072 Spanish, and 35,800 Valencian samples.
Intended Uses
This model can be used for:
- Instruction following in Valencian, Spanish, and English with improved alignment to human preferences
- Chat and conversational applications requiring high-quality multilingual responses
- Text generation with task-specific prompting and improved output quality
- Domain-specific applications in administrative, legal, or tourism contexts
Note: As an aligned instruction-tuned model, it is designed to follow user prompts and generate helpful, safe responses. It is not intended for use as a factual knowledge base. The DPO alignment improves response quality and preference alignment.
How to Use
import torch
from transformers import pipeline, AutoTokenizer
model_id = "gplsi/Aitana-2B-SI-Instruct-Aligned"
tokenizer = AutoTokenizer.from_pretrained(model_id)
generator = pipeline(
"text-generation",
model=model_id,
tokenizer=tokenizer,
torch_dtype=torch.bfloat16,
device_map="auto",
)
text = "Explica què són les Corts Valencianes i quina funció tenen."
result = generator(text, do_sample=True, top_k=10, max_new_tokens=100)
print(result[0]['generated_text'])
text = "Describe las principales funciones del gobierno autonómico valenciano."
result = generator(text, do_sample=True, top_k=10, max_new_tokens=100)
print(result[0]['generated_text'])
text = "Explain the role of tourism in the Valencian Community economy."
result = generator(text, do_sample=True, top_k=10, max_new_tokens=100)
print(result[0]['generated_text'])
Evaluation
In the following tables, we present the results obtained with different benchmarks from lm-evaluation-harness in comparison with Salamandra-2B-Instruct, and Aitana-2B-S-Instruct-Aligned. The results reflect the DPO-aligned instruction-tuned performance.
Valencian
Classification Benchmarks
Table with columns: Dataset, Lang., Task, Metric, Salamandra-2B-Instruct, Aitana-2B-S-Instruct-Aligned (v0.1), Aitana-2B-SI-Instruct-Aligned (v0.1)| Dataset | Lang. | Task | Metric | Salamandra-2B-Instruct | Aitana-2B-S-Instruct-Aligned (v0.1) | Aitana-2B-SI-Instruct-Aligned (v0.1) |
|---|
| XNLI | va | Natural Language Inference | acc | 0.520 | 0.514 | 0.485 |
Generation Benchmarks
Table with columns: Dataset, Lang., Task, Metric, Salamandra-2B-Instruct, Aitana-2B-S-Instruct-Aligned (v0.1), Aitana-2B-SI-Instruct-Aligned (v0.1)| Dataset | Lang. | Task | Metric | Salamandra-2B-Instruct | Aitana-2B-S-Instruct-Aligned (v0.1) | Aitana-2B-SI-Instruct-Aligned (v0.1) |
|---|
| Cocoteros | va | Reading Comprehension | bleu | 2.796 | 3.612 | 4.223 |
| Phrases ca-va | va-ca | Translation - Adaptation | bleu | 58.425 |
Catalan
Classification Benchmarks
Table with columns: Dataset, Lang., Task, Metric, Salamandra-2B-Instruct, Aitana-2B-S-Instruct-Aligned (v0.1), Aitana-2B-SI-Instruct-Aligned (v0.1)| Dataset | Lang. | Task | Metric | Salamandra-2B-Instruct | Aitana-2B-S-Instruct-Aligned (v0.1) | Aitana-2B-SI-Instruct-Aligned (v0.1) |
|---|
| Belebele Cat_latn | ca | Reading Comprehension | acc | 0.287 | 0.248 | 0.319 |
| COPA | ca | Commonsense Reasoning | acc | 0.708 |
Generation Benchmarks
Table with columns: Dataset, Lang., Task, Metric, Salamandra-2B-Instruct, Aitana-2B-S-Instruct-Aligned (v0.1), Aitana-2B-SI-Instruct-Aligned (v0.1)| Dataset | Lang. | Task | Metric | Salamandra-2B-Instruct | Aitana-2B-S-Instruct-Aligned (v0.1) | Aitana-2B-SI-Instruct-Aligned (v0.1) |
|---|
| Cabreu abstractive | ca | Summarization | bleu | 7.610 | 7.703 | 8.837 |
| Cabreu extractive | ca | Summarization | bleu | |
Spanish
Classification Benchmarks
Table with columns: Dataset, Lang., Task, Metric, Salamandra-2B-Instruct, Aitana-2B-S-Instruct-Aligned (v0.1), Aitana-2B-SI-Instruct-Aligned (v0.1)| Dataset | Lang. | Task | Metric | Salamandra-2B-Instruct | Aitana-2B-S-Instruct-Aligned (v0.1) | Aitana-2B-SI-Instruct-Aligned (v0.1) |
|---|
| Belebele | es | Reading Comprehension | acc | 0.268 | 0.244 | 0.285 |
| PAWS | es | Paraphrasing | acc | 0.566 |
Generation Benchmarks
Table with columns: Dataset, Lang., Task, Metric, Salamandra-2B-Instruct, Aitana-2B-S-Instruct-Aligned (v0.1), Aitana-2B-SI-Instruct-Aligned (v0.1)| Dataset | Lang. | Task | Metric | Salamandra-2B-Instruct | Aitana-2B-S-Instruct-Aligned (v0.1) | Aitana-2B-SI-Instruct-Aligned (v0.1) |
|---|
| Cocoteros | es | Reading Comprehension | bleu | 3.308 | 3.141 | 3.670 |
| XLSum | es | Summarization | bleu | 1.695 |
English
Classification Benchmarks
Table with columns: Dataset, Lang., Task, Metric, Salamandra-2B-Instruct, Aitana-2B-S-Instruct-Aligned (v0.1), Aitana-2B-SI-Instruct-Aligned (v0.1)| Dataset | Lang. | Task | Metric | Salamandra-2B-Instruct | Aitana-2B-S-Instruct-Aligned (v0.1) | Aitana-2B-SI-Instruct-Aligned (v0.1) |
|---|
| Arc Challenge | en | Question Answering | acc | 0.354 | 0.363 | 0.372 |
| Arc Easy | en | Question Answering | acc | 0.681 |
Judge Evaluation
The model was also evaluated using an LLM-as-judge approach across different task categories. The scores below represent the average rating (1-5 scale, 5 being best) and standard deviation for each task category, comparing Aitana-2B-SI-Instruct-Aligned against Salamandra-2B-Instruct and Aitana-2B-S-Instruct-Aligned.
Table with columns: Task Category, Salamandra-2B-Instruct, Aitana-2B-S-Instruct-Aligned (v0.1), Aitana-2B-SI-Instruct-Aligned (v0.1)| Task Category | Salamandra-2B-Instruct | Aitana-2B-S-Instruct-Aligned (v0.1) | Aitana-2B-SI-Instruct-Aligned (v0.1) |
|---|
| CommonSense reasoning | 2.277 / 1.151 | 2.737 / 1.140 | 2.969 / 1.086 |
| Maths | 1.060 / 0.124 | 1.123 / 0.249 | 1.191 / 0.349 |
| Paraphrasing | 3.518 / 1.308 | 3.460 / 1.088 | 3.472 / 0.959 |
| Reading comprehension | |
The DPO-aligned model shows a notable improvement in overall average score (2.787) compared to Aitana-2B-S-Instruct-Aligned (v0.1) (2.649) and Salamandra-2B-Instruct (2.599) with particular gains in CommonSense reasoning, reading comprehension and summarization. The aligned model also shows tighter standard deviations in several categories, indicating more consistent quality responses.
Author
The model has been developed by the Language and Information Systems Group (GPLSI) and the Centro de Inteligencia Digital (CENID), both part of the University of Alicante (UA), as part of their ongoing research in Natural Language Processing (NLP).
Funding
This work is funded by the Ministerio para la Transformación Digital y de la Función Pública, co-financed by the EU – NextGenerationEU, within the framework of the project Desarrollo de Modelos ALIA. This work has also been partially supported by Project HEART-NLP (PID2024-156263OB-C22).
Acknowledgments
We would like to express our gratitude to all individuals and institutions that have contributed to the development of this work.
Special thanks to:
We also acknowledge the financial, technical, and scientific support of the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Desarrollo de Modelos ALIA, whose contribution has been essential to the completion of this research.
License
Apache License, Version 2.0
Disclaimer
This model is intended for general purposes and is available under a permissive Apache License 2.0. Be aware that the model may have biases and/or undesirable outputs. Users deploying systems based on this model are responsible for mitigating risks and complying with applicable AI regulations.
Reference
@misc{gplsi-aitana-2B-SI-Instruct-Aligned,
author = {Galiano, Santiago and Sepúlveda-Torres, Robiert and Martínez-Murillo, Iván and Estevanell-Valladares, Ernesto L. and Grande, Eduardo and Consuegra-Ayala, Juan Pablo and Miró Maestre, María and Canal-Esteve, Miquel and Bonora, Mar and Gutierrez, Yoan and Abreu Salas, José Ignacio and Lloret, Elena and Montoyo, Andrés and Muñoz-Guillena, Rafael and Palomar, Manuel},
title = {Aitana-2B-SI-Instruct-Aligned: DPO-aligned instruction-tuned model for Valencian, Spanish and English},
year = {2026},
institution = {Language and Information Systems Group (GPLSI) and Centro de Inteligencia Digital (CENID), University of Alicante (UA)},
howpublished = {\url{https://huggingface.co/gplsi/Aitana-2B-SI-Instruct-Aligned}},
note = {Accessed: 2026-05-11}
}
Copyright © 2026 Language and Information Systems Group (GPLSI) and Centro de Inteligencia Digital (CENID), University of Alicante (UA). Distributed under the Apache License 2.0.
