Table of Contents

• Model Description
• Alignment Details
• Training Data
• Intended Uses
• How to Use
• Evaluation
• Additional Information

Model Description

Aitana-2B-S-Instruct-Aligned extends the Aitana-2B-S-Instruct instruction-tuned model with Direct Preference Optimization (DPO) alignment. This additional training stage improves the model's ability to generate helpful, high-quality responses that better align with human preferences while maintaining its strong multilingual capabilities.

Alignment Details

The model was aligned using Direct Preference Optimization (DPO) with the following configuration:

The DPO alignment was performed using curated preference pairs that teach the model to prefer more helpful, accurate, and well-structured responses.

Training Data

The base instruction model was trained on the ALIA Instruction/v12 dataset. This DPO-aligned variant was further aligned using the Alignment/v8 dataset, composed of the following preference data:

The alignment data focused on English, Spanish, and Valencian preference pairs, with the distribution: 80,308 English, 30,072 Spanish, and 35,800 Valencian samples.

Intended Uses

This model can be used for:

• Instruction following in Valencian, Spanish, and English with improved alignment to human preferences
• Chat and conversational applications requiring high-quality multilingual responses
• Text generation with task-specific prompting and improved output quality
• Domain-specific applications in administrative, legal, or tourism contexts

Note: As an aligned instruction-tuned model, it is designed to follow user prompts and generate helpful, safe responses. It is not intended for use as a factual knowledge base. The DPO alignment improves response quality and preference alignment.

How to Use

Transformers

python
Copy code
import torch
from transformers import pipeline, AutoTokenizer

model_id = "gplsi/Aitana-2B-S-Instruct-Aligned"
tokenizer = AutoTokenizer.from_pretrained(model_id)
generator = pipeline(
    "text-generation",
    model=model_id,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
# Valencian example
text = "Explica què són les Corts Valencianes i quina funció tenen."
result = generator(text, do_sample=True, top_k=10, max_new_tokens=100)
print(result[0]['generated_text'])
# Spanish example
text = "Describe las principales funciones del gobierno autonómico valenciano."
result = generator(text, do_sample=True, top_k=10, max_new_tokens=100)
print(result[0]['generated_text'])
# English example
text = "Explain the role of tourism in the Valencian Community economy."
result = generator(text, do_sample=True, top_k=10, max_new_tokens=100)
print(result[0]['generated_text'])

Evaluation

In the following tables, we present the results obtained with different benchmarks from lm-evaluation-harness in comparison with Salamandra-2B-Instruct. The results reflect the DPO-aligned instruction-tuned performance.

Valencian

Classification Benchmarks

Generation Benchmarks

Catalan

Classification Benchmarks

Generation Benchmarks

Spanish

Classification Benchmarks

Generation Benchmarks

English

Classification Benchmarks

Judge Evaluation

The model was also evaluated using an LLM-as-judge approach across different task categories. The scores below represent the average rating (1-5 scale, 5 being best) and standard deviation for each task category, comparing Aitana-2B-S-Instruct-Aligned-v0.1 against Salamandra-2B-Instruct.

The DPO-aligned model shows a notable improvement in overall average score (2.649 vs 2.599) compared to Salamandra-2B-Instruct, with particular gains in CommonSense reasoning and Maths. The aligned model also shows tighter standard deviations in several categories, indicating more consistent quality responses.

Additional Information

Author

The model has been developed by the Language and Information Systems Group (GPLSI) and the Centro de Inteligencia Digital (CENID), both part of the University of Alicante (UA), as part of their ongoing research in Natural Language Processing (NLP).

Funding

This work is funded by the Ministerio para la Transformación Digital y de la Función Pública, co-financed by the EU – NextGenerationEU, within the framework of the project Desarrollo de Modelos ALIA. This work has also been partially supported by Project HEART-NLP (PID2024-156263OB-C22).

Acknowledgments

We would like to express our gratitude to all individuals and institutions that have contributed to the development of this work. Special thanks to:

• Language Technologies Laboratory at Barcelona Supercomputing Center
• Centro Vasco de Tecnología de la Lengua (HiTZ)
• Centro Singular de Investigación en Tecnologías Inteligentes (CiTIUS)
• Sistemas Inteligentes de Acceso a la Información (SINAI)
• Instituto Universitario de Investigación Informática (IUII)
• Leonardo HPC System
• European supercomputing ecosystem (EUROHPC)

We also acknowledge the financial, technical, and scientific support of the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Desarrollo de Modelos ALIA, whose contribution has been essential to the completion of this research.

License

Apache License, Version 2.0

Disclaimer

This model is intended for general purposes and is available under a permissive Apache License 2.0. Be aware that the model may have biases and/or undesirable outputs. Users deploying systems based on this model are responsible for mitigating risks and complying with applicable AI regulations.

Reference

bibtex
Copy code
@misc{gplsi-aitana-2B-S-Instruct-Aligned,
  author       = {Galiano, Santiago and Sepúlveda-Torres, Robiert and Martínez-Murillo, Iván and Grande, Eduardo and Consuegra-Ayala, Juan Pablo and Miró Maestre, María and Canal-Esteve, Miquel and Bonora, Mar and Gutierrez, Yoan and Abreu Salas, José Ignacio and Lloret, Elena and Montoyo, Andrés and Muñoz-Guillena, Rafael and Palomar, Manuel},
  title        = {Aitana 2B Instruct Aligned: DPO-aligned instruction-tuned model for Valencian, Spanish and English},
  year         = {2026},
  institution  = {Language and Information Systems Group (GPLSI) and Centro de Inteligencia Digital (CENID), University of Alicante (UA)},
  howpublished = {\url{https://huggingface.co/gplsi/Aitana-2B-S-Instruct-Aligned}},
  note         = {Accessed: 2026-05-11}
}

---

Copyright © 2026 Language and Information Systems Group (GPLSI) and Centro de Inteligencia Digital (CENID), University of Alicante (UA). Distributed under the Apache License 2.0.