gplsi

Aitana-TA-7B-S

README

License: apache-2.0

Model Description
Training Details
Intended Uses
How to Use
Evaluation
Additional Information

Model Description

Table with columns: Property, Value
Property	Value
Base Model	BSC-LT/salamandraTA-7b-instruct
Architecture	Transformer decoder-only
Parameters	~7.76B
Languages	Valencian and Spanish
License	Apache 2.0
Finetuned from model:	This model follows the same instruction pattern as SalamandraTA-7b-Instructed. The only adaptation was focusing the instructions on translation between Valencian and Spanish, in both directions.

Training Details

Training Data

The data comes from L'Associació de Mitjans d'Informació i Comunicació (AMIC), from which we built a parallel sentence-level dataset. This dataset was specifically created to align Spanish and Valencian sentences, ensuring high-quality parallel examples for training and evaluation.

== Language pairs== Valencian -> Spanish: 738777 (mean_src_len=165.6, mean_tgt_len=168.6)

Training Hyperparameters

Training regime:

epochs: 1
learning_rate: 1e-5
beta1: 0.9
beta2: 0.99
weight_decay: 0
global_batch_size: 64
micro_batch_size: 2
log_interval: 5
save_interval: 5
lr_warmup_steps: 100
max_seq_length: 2048

How to use

You can translate between Spanish to Valencian. The instruction-following model uses the commonly adopted ChatML template:

markdown
<|im_start|>system
{SYSTEM PROMPT}<|im_end|>
<|im_start|>user
{USER PROMPT}<|im_end|>
<|im_start|>assistant
{MODEL RESPONSE}<|im_end|>
<|im_start|>user
[...]

The easiest way to apply it is by using the tokenizer's built-in functions, as shown in the following snippet.

python
from datetime import datetime
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model_id = "gplsi/Aitana-TA-7b-S"

source = 'Spanish'
target = 'Valencian'
sentence = "La inteligencia artificial está transformando el mundo."
 
text = f"Translate the following text from {source} into {target}.\n{source}: {sentence} \n{target}:"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype=torch.bfloat16
  )

message = [ { "role": "user", "content": text } ]

prompt = tokenizer.apply_chat_template(
    message,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
input_length = inputs.shape[1]
outputs = model.generate(input_ids=inputs.to(model.device), 
                         max_new_tokens=400,
                         early_stopping=True,
                         num_beams=5)

print(tokenizer.decode(outputs[0, input_length:], skip_special_tokens=True))
# La intel·ligència artificial està transformant el món.

Using this template, each turn is preceded by a <|im_start|> delimiter and the role of the entity (either user, for content supplied by the user, or assistant for LLM responses), and finished with the <|im_end|> token.

Evaluation

Testing Data

Our objective was to evaluate translation between Spanish and Valencian. To test the model, we used the Phrases task from IberoBench.

Metrics

For evaluation, we relied on a set of standard translation metrics: COMET, BLEU, TER, and ChrF.

Results on Phrases tasks

Table with columns: Task, Metric, Aitana-TA-7b-S, salamandraTA-7b-instruct
Task	Metric	Aitana-TA-7b-S	salamandraTA-7b-instruct
phrases_es-va	BLEU	63.35	62.40
phrases_va-es	BLEU	81.15	75.49
phrases_va-ca	BLEU	79.24	82.07
phrases_ca-va	BLEU	79.95	76.53

Technical Specifications

Hardware and Software

For training, we used custom code developed on top of the PyTorch Lightning framework (Fabric Lightning), enabling model sharding across multiple GPUs through Fully Sharded Data Parallel (FSDP).

Compute Infrastructure

This model was trained on NVIDIA DGX systems equipped with A100 GPUs, which enabled efficient large-scale training. For this model, we used 4-A100 GPUs.

Additional Information

Author

The model has been developed by the Language and Information Systems Group (GPLSI) and the Centro de Inteligencia Digital (CENID), both part of the University of Alicante (UA), as part of their ongoing research in Natural Language Processing (NLP).

Funding

This work is funded by the Ministerio para la Transformación Digital y de la Función Pública, co-financed by the EU – NextGenerationEU, within the framework of the project Desarrollo de Modelos ALIA. This work has also been partially supported by Project HEART-NLP (PID2024-156263OB-C22).

Acknowledgments

We would like to express our gratitude to all individuals and institutions that have contributed to the development of this work.

Special thanks to:

We also acknowledge the financial, technical, and scientific support of the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Desarrollo de Modelos ALIA, whose contribution has been essential to the completion of this research.

License

Apache License, Version 2.0

Disclaimer

This model has been developed and instruction-tuned specifically for translation between Spanish and Valencian. Its use outside of translation tasks is not recommended, as performance and reliability have not been evaluated for other natural language processing applications. The authors are not responsible for potential errors, misinterpretations, or inappropriate use of the model beyond its intended purpose.

Reference

If you use this model in your research or work, please cite it as follows:

bibtex
@misc{gplsi-aitama-ta-7b-s,
  author       = {Sepúlveda-Torres, Robiert and Galeano, Santiago and Miró Maestre, María and Martínez-Murillo, Iván and Grande, Eduardo and Canal-Esteve, Miquel and Estevanell-Valladares, Ernesto L. and Yáñez-Romero, Fabio and Consuegra-Ayala, Juan Pablo and Bonora, Mar and Gutierrez, Yoan and Abreu Salas, José Ignacio and Lloret, Elena and Montoyo, Andrés and Muñoz-Guillena and Palomar, Manuel},
  title        = {Aitana-TA-7b-S: Translation model from Spanish to Valencian},
  year         = {2025},
  institution  = {Language and Information Systems Group (GPLSI) and Centro de Inteligencia Digital (CENID), University of Alicante (UA)},
  howpublished = {\url{https://huggingface.co/gplsi/Aitana-TA-7b-S}},
  note         = {Accessed: 2025-10-03}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

gplsi

Model Tree

Base

BSC-LT/salamandraTA-7b-instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Container

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Description
Training Details
Intended Uses
How to Use
Evaluation
Additional Information

Model Description

Table with columns: Property, Value
Property	Value
Base Model	BSC-LT/salamandraTA-7b-instruct
Architecture	Transformer decoder-only
Parameters	~7.76B
Languages	Valencian and Spanish
License	Apache 2.0
Finetuned from model:	This model follows the same instruction pattern as SalamandraTA-7b-Instructed. The only adaptation was focusing the instructions on translation between Valencian and Spanish, in both directions.

Training Details

Training Data

== Language pairs== Valencian -> Spanish: 738777 (mean_src_len=165.6, mean_tgt_len=168.6)

Training Hyperparameters

Training regime:

epochs: 1
learning_rate: 1e-5
beta1: 0.9
beta2: 0.99
weight_decay: 0
global_batch_size: 64
micro_batch_size: 2
log_interval: 5
save_interval: 5
lr_warmup_steps: 100
max_seq_length: 2048

How to use

You can translate between Spanish to Valencian. The instruction-following model uses the commonly adopted ChatML template:

markdown
<|im_start|>system
{SYSTEM PROMPT}<|im_end|>
<|im_start|>user
{USER PROMPT}<|im_end|>
<|im_start|>assistant
{MODEL RESPONSE}<|im_end|>
<|im_start|>user
[...]

The easiest way to apply it is by using the tokenizer's built-in functions, as shown in the following snippet.

python
from datetime import datetime
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model_id = "gplsi/Aitana-TA-7b-S"

source = 'Spanish'
target = 'Valencian'
sentence = "La inteligencia artificial está transformando el mundo."
 
text = f"Translate the following text from {source} into {target}.\n{source}: {sentence} \n{target}:"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype=torch.bfloat16
  )

message = [ { "role": "user", "content": text } ]

prompt = tokenizer.apply_chat_template(
    message,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
input_length = inputs.shape[1]
outputs = model.generate(input_ids=inputs.to(model.device), 
                         max_new_tokens=400,
                         early_stopping=True,
                         num_beams=5)

print(tokenizer.decode(outputs[0, input_length:], skip_special_tokens=True))
# La intel·ligència artificial està transformant el món.

Evaluation

Testing Data

Our objective was to evaluate translation between Spanish and Valencian. To test the model, we used the Phrases task from IberoBench.

Metrics

For evaluation, we relied on a set of standard translation metrics: COMET, BLEU, TER, and ChrF.

Results on Phrases tasks

Table with columns: Task, Metric, Aitana-TA-7b-S, salamandraTA-7b-instruct
Task	Metric	Aitana-TA-7b-S	salamandraTA-7b-instruct
phrases_es-va	BLEU	63.35	62.40
phrases_va-es	BLEU	81.15	75.49
phrases_va-ca	BLEU	79.24	82.07
phrases_ca-va	BLEU	79.95	76.53

Technical Specifications

Hardware and Software

For training, we used custom code developed on top of the PyTorch Lightning framework (Fabric Lightning), enabling model sharding across multiple GPUs through Fully Sharded Data Parallel (FSDP).

Compute Infrastructure

This model was trained on NVIDIA DGX systems equipped with A100 GPUs, which enabled efficient large-scale training. For this model, we used 4-A100 GPUs.

Additional Information

Author

Funding

Acknowledgments

We would like to express our gratitude to all individuals and institutions that have contributed to the development of this work.

Special thanks to:

License

Apache License, Version 2.0

Disclaimer

Reference

If you use this model in your research or work, please cite it as follows:

bibtex
@misc{gplsi-aitama-ta-7b-s,
  author       = {Sepúlveda-Torres, Robiert and Galeano, Santiago and Miró Maestre, María and Martínez-Murillo, Iván and Grande, Eduardo and Canal-Esteve, Miquel and Estevanell-Valladares, Ernesto L. and Yáñez-Romero, Fabio and Consuegra-Ayala, Juan Pablo and Bonora, Mar and Gutierrez, Yoan and Abreu Salas, José Ignacio and Lloret, Elena and Montoyo, Andrés and Muñoz-Guillena and Palomar, Manuel},
  title        = {Aitana-TA-7b-S: Translation model from Spanish to Valencian},
  year         = {2025},
  institution  = {Language and Information Systems Group (GPLSI) and Centro de Inteligencia Digital (CENID), University of Alicante (UA)},
  howpublished = {\url{https://huggingface.co/gplsi/Aitana-TA-7b-S}},
  note         = {Accessed: 2025-10-03}
}

Aitana-TA-7B-S

README

Table of Contents

Model Description

Training Details

Training Data

Training Hyperparameters

How to use

Evaluation

Testing Data

Metrics

Results on Phrases tasks

Technical Specifications

Hardware and Software

Compute Infrastructure

Additional Information

Author

Funding

Acknowledgments

License

Disclaimer

Reference

Explore FriendliAI today

README

Table of Contents

Model Description

Training Details

Training Data

Training Hyperparameters

How to use

Evaluation

Testing Data

Metrics

Results on Phrases tasks

Technical Specifications

Hardware and Software

Compute Infrastructure

Additional Information

Author

Funding

Acknowledgments

License

Disclaimer

Reference