Model Details
Model Description
Latxa is a family of Large Language Models (LLMs) for Basque. Languages with few resources get much less attention from model developers than English, and models trained mostly on English data perform poorly in them. Latxa models narrow this gap for Basque. Latxa 3 has been adapted following the methodology described in Sainz et al. (2025), on top of Qwen3.5-122B-A10B.
- Developed by: HiTZ Research Center & IXA Research group (University of the Basque Country UPV/EHU)
- Funded by: Ikergaitu project (Basque Government)
- Model type: Multimodal (vision-language) instruct model, Mixture-of-Experts (256 experts, 8 routed + 1 shared per token)
- Parameters: 122B total, ~10B active
- Context length: 262,144 tokens
- Language(s) (NLP): Basque (eu); other languages supported by the base model (e.g. English) are retained but not the focus of the adaptation
- License: Apache 2.0
- Finetuned from model: Qwen/Qwen3.5-122B-A10B
- Contact: hitz@ehu.eus
Quantization
The weights are quantized to FP8 with llm-compressor (compressed-tensors format, FP8_DYNAMIC scheme):
- Weights: FP8, symmetric, per-channel (static).
- Activations: FP8, symmetric, per-token (dynamic).
- Kept in BF16:
lm_head, embeddings, MoE router gates, linear-attention conv1d/in_proj_a/in_proj_b, the vision encoder, and the MTP projection.
This roughly halves memory use compared to BF16 (~125GB of weights). The model runs on 2×A100 80GB GPUs (our setup) or on a single H200 (141GB). See Getting Started.
Getting Started
We recommend serving the model with vLLM, which supports the compressed-tensors FP8 format natively:
2×A100 80GB (the setup we use):
vllm serve HiTZ/Latxa-3-122B-A10B-FP8 \ --served-model-name Latxa-3 \ --tensor-parallel-size 2 \ --enable-expert-parallel \ --max-model-len 262144 \ --gpu-memory-utilization 0.95 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3
[!NOTE]
A100 GPUs have no native FP8 support, so vLLM uses weight-only FP8 kernels (Marlin) on them. Hopper GPUs (H100/H200) and newer run full FP8 (W8A8) and will be faster.
Single H200 (141GB):
vllm serve HiTZ/Latxa-3-122B-A10B-FP8 \ --served-model-name Latxa-3 \ --max-model-len 131072 \ --gpu-memory-utilization 0.95 \ --kv-cache-dtype fp8 \ --max-num-seqs 16 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3
On a single H200 the weights take up most of the memory, which leaves only a few GB for the KV cache. Only 12 of the 48 layers use full attention, so the cache is small (~12KB per token with an FP8 KV cache). Still, we recommend limiting the context length and the number of concurrent sequences. With low concurrency you can raise --max-model-len to the full 262144 tokens. If you hit OOM errors at startup, lower --max-num-seqs or --max-model-len.
Then query it with any OpenAI-compatible client:
from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") messages = [ { "role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}}, {"type": "text", "text": "Zer ikusten da irudi honetan?"}, ], },] response = client.chat.completions.create( model="Latxa-3", messages=messages, temperature=0.5, top_p=0.9, presence_penalty=0.0, max_tokens=65536, extra_body={"top_k": 20, "min_p": 0.0, "repetition_penalty": 1.0},)print(response.choices[0].message.content)
It can also be used with the transformers pipeline:
from transformers import pipeline pipe = pipeline("image-text-to-text", model="HiTZ/Latxa-3-122B-A10B-FP8", device_map="auto")messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "Zer ikusten da irudi honetan?"}, ], },]print(pipe(text=messages, max_new_tokens=1024))
[!TIP]
We recommend the following sampling parameters:
Table with columns: Parameter, Value| Parameter | Value |
|---|
| temperature | 0.5 |
| top_p | 0.9 |
| top_k | 20 |
| min_p | 0.0 |
| presence_penalty | 0.0 |
| repetition_penalty | 1.0 |
| max_tokens | 65536 |
max_tokens can be increased if needed (e.g. for long reasoning traces). Support for these parameters varies by inference framework.
[!WARNING]
Latxa 3 currently supports thinking mode only. Unlike the base Qwen3.5 model, it has not been trained to answer without reasoning. Do not disable thinking (e.g. with enable_thinking: False in the chat template), because output quality is not guaranteed. With --reasoning-parser qwen3, vLLM returns the reasoning trace separately, in the reasoning_content field.
Uses
Latxa models are intended to be used with Basque data. Performance in other languages is not guaranteed.
Direct Use
Latxa 3 is instruction-tuned and can follow instructions or work as a chat assistant, including on tasks that involve images.
Out-of-Scope Use
The model is not intended for malicious activities, such as harming others or violating human rights. Any downstream application must comply with current laws and regulations. Irresponsible use in production environments without proper risk assessment and mitigation is also discouraged.
Bias, Risks, and Limitations
To reduce potentially disturbing or harmful content, Latxa has been trained on carefully selected and processed data, which comes mainly from local media, national/regional newspapers, encyclopedias and blogs (see Latxa Corpus v2). Still, the model is based on Qwen3.5 and may carry the same biases, risks and limitations.
As with any quantized model, the FP8 version may behave slightly differently from the original BF16 weights.
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
Training Details
Training Data
Latxa 3 was trained on Latxa Corpus v2, following the adaptation recipe of Sainz et al. (2025). Before training the large models, we ran development experiments on smaller Qwen3.5 models. These compared training configurations and tested whether to add reasoning traces to the instruction data and whether to include multimodal data. The best configuration was then used to train three Qwen3.5 models (27B, 35B-A3B and 122B-A10B). The 122B-A10B model performed best, so it received an additional training pass. That model is Latxa 3.
Training Procedure
The model was trained with Swift/Megatron on the MareNostrum 5 Accelerated Partition (BSC). Jobs ran on 32 and 64 nodes (128 and 256 NVIDIA H100 64GB GPUs).
Evaluation
We compare Latxa 3 with its base model, Qwen3.5-122B-A10B. As a reference, we also include Latxa 2 (Llama-3.1-70B), adapted with the same recipe. The largest gains are in Basque language proficiency (+12.4 points) and knowledge about Basque culture (+21.2 points). General knowledge stays the same or improves slightly.
All datasets are evaluated as text generation: the model produces an answer, and that answer is scored. Multiple-choice tasks are scored by checking the chosen option. Open-ended questions (marked open) are scored by an LLM judge, which compares the model's answer with a reference answer.
[!NOTE]
These results were obtained with the BF16 weights.
Basque language proficiency (accuracy, %)
Table with columns: Task, Latxa 2 (Llama-3.1-70B), Qwen3.5-122B-A10B, Latxa 3| Task | Latxa 2 (Llama-3.1-70B) | Qwen3.5-122B-A10B | Latxa 3 |
|---|
| EusProficiency | 70.8 | 61.1 | 82.4 |
| EusReading | 80.4 | 84.4 | 87.8 |
| BAT (Batua norms) | 75.9 | 69.3 | 81.8 |
| Average | 75.7 | 71.6 | 84.0 |
Basque knowledge (accuracy, %)
Table with columns: Task, Latxa 2 (Llama-3.1-70B), Qwen3.5-122B-A10B, Latxa 3| Task | Latxa 2 (Llama-3.1-70B) | Qwen3.5-122B-A10B | Latxa 3 |
|---|
| EusExams v2 | 76.2 | 81.7 | 84.9 |
| EusTrivia | 84.1 | 77.8 | 87.8 |
| EusTrivia (open) | 59.9 | 47.6 | 63.5 |
| MMLU-Pro-eus | 54.0 | 76.8 | 77.2 |
| MMLU-Pro-eus (open) | 42.3 | 67.8 | 68.5 |
| Average | 63.3 | 70.3 | 76.4 |
Cultural knowledge (accuracy, %). Local questions are about Basque culture. Global questions are about general knowledge and serve as a control.
Table with columns: Task, Latxa 2 (Llama-3.1-70B), Qwen3.5-122B-A10B, Latxa 3| Task | Latxa 2 (Llama-3.1-70B) | Qwen3.5-122B-A10B | Latxa 3 |
|---|
| Local, multiple choice, Basque | 81.7 | 69.7 | 85.8 |
| Local, multiple choice, English | 77.8 | 66.9 | 83.6 |
| Local, open-ended, Basque | 62.5 | 31.4 | 60.3 |
| Local, open-ended, English | 52.4 | 28.0 | 51.1 |
| Local average | 68.6 | 49.0 | 70.2 |
| Global, multiple choice, Basque | 91.9 | 93.6 | 94.4 |
| Global, multiple choice, English | 92.2 | 93.6 | 93.3 |
| Global, open-ended, Basque | 73.2 | 74.9 | 77.2 |
| Global, open-ended, English | 74.4 | 80.2 | 78.5 |
| Global average | 82.9 | 85.6 | 85.9 |
Machine translation (chrF). Sentence level uses FLORES and document level uses ALIA DocMT.
Table with columns: Direction, Latxa 2 (Llama-3.1-70B), Qwen3.5-122B-A10B, Latxa 3| Direction | Latxa 2 (Llama-3.1-70B) | Qwen3.5-122B-A10B | Latxa 3 |
|---|
| FLORES en → eu | 60.5 | 57.7 | 60.2 |
| FLORES es → eu | 54.7 | 52.3 | 54.5 |
| FLORES into Basque (avg.) | 57.6 | 55.0 | 57.3 |
| FLORES eu → en | 62.2 | 60.5 | 61.4 |
| FLORES eu → es | 50.7 | 50.5 | 51.1 |
| FLORES out of Basque (avg.) | 56.5 | 55.5 | 56.3 |
| DocMT en → eu | 62.0 | 62.3 | 65.6 |
| DocMT es → eu | 70.2 | 66.5 | 73.4 |
| DocMT into Basque (avg.) | 66.1 | 64.4 | 69.5 |
| DocMT eu → en | 67.0 | 66.3 | 67.3 |
| DocMT eu → es | 68.4 | 71.9 | 73.6 |
| DocMT out of Basque (avg.) | 67.7 | 69.1 | 70.5 |
In all tables, bold marks the best score in each row.
[!WARNING]
The paper describing this model is in preparation. Further training and evaluation details will be released with it.
Environmental Impact
Energy use is estimated from GPU hours, assuming an average draw of 0.7 kW per H100 GPU (its nominal TDP) and a grid emission factor of 0.155 kg CO2eq/kWh.
- Hardware Type: NVIDIA H100 64GB (4 GPUs per node, up to 64 nodes)
- Compute cluster: MareNostrum 5 ACC (Barcelona Supercomputing Center)
- Compute Region: Spain
- Hours used (total GPU hours): 34,832.63
- Energy: 24,382.84 kWh
- Carbon Emitted: 3,779.34 kg CO2eq
Citation
@inproceedings{sainz-etal-2025-instructing, title = "Instructing Large Language Models for Low-Resource Languages: A Systematic Study for {B}asque", author = "Sainz, Oscar and Perez, Naiara and Etxaniz, Julen and Fernandez de Landa, Joseba and Aldabe, Itziar and Garc{\'i}a-Ferrero, Iker and Zabala, Aimar and Azurmendi, Ekhi and Rigau, German and Agirre, Eneko and Artetxe, Mikel and Soroa, Aitor", booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing", year = "2025", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2025.emnlp-main.1484/", doi = "10.18653/v1/2025.emnlp-main.1484", pages = "29124--29148"}
Acknowledgements
This work has been partially supported by the Basque Government (Research group funding IT1570-22 and IKER-GAITU project). Latxa 3 was trained on the MareNostrum 5 supercomputer at the Barcelona Supercomputing Center (BSC), through the EuroHPC Joint Undertaking AI Factory Large Scale Access project EHPC-AIF-2026LS03-009. EJIE (Basque Government Informatic Society) led that project together with HiTZ (UPV/EHU). We thank the BSC support team for their help in setting up the software environment.