Adapter Description
Table with columns: Property, Value| Property | Value |
|---|
| Base Model | google/gemma-2-9b-it |
| Translation Direction | English → Yoruba |
| LoRA Rank (r) | 64 |
| LoRA Alpha | 128 |
| Training Method | QLoRA (4-bit quantization) |
| Domain | Scientific/Academic texts |
Why LoRA?
LoRA (Low-Rank Adaptation) enables efficient fine-tuning by training only a small number of additional parameters. This adapter adds only ~32.0M parameters to the base model while achieving strong translation performance.
Evaluation Results
Performance on the AfriScience-MT test set:
Table with columns: Split, BLEU, chrF, SSA-COMET| Split | BLEU | chrF | SSA-COMET |
|---|
| Test | - | - | - |
Metrics explanation:
- BLEU: Measures n-gram overlap with reference translations (0-100, higher is better)
- chrF: Character-level F-score, robust for morphologically rich languages (0-100, higher is better)
- SSA-COMET: Neural metric trained for Sub-Saharan African languages, shown as percentage (0-100, higher is better) (McGill-NLP/ssa-comet-stl)
Usage
Quick Start
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
)
base_model = AutoModelForCausalLM.from_pretrained(
"google/gemma-2-9b-it",
quantization_config=bnb_config,
device_map="auto",
torch_dtype=torch.bfloat16,
)
tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b-it")
adapter_name = "dsfsi/gemma_2_9b_it-lora-r64-eng-yor"
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()
source_text = "Climate change significantly impacts agricultural productivity in sub-Saharan Africa."
instruction = "Translate the following English scientific text to Yoruba."
messages = [{"role": "user", "content": f"{instruction}\n\n{source_text}"}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=256,
num_beams=5,
early_stopping=True,
pad_token_id=tokenizer.pad_token_id,
)
generated = outputs[0][inputs["input_ids"].shape[1]:]
translation = tokenizer.decode(generated, skip_special_tokens=True)
print(translation)
Without Quantization (Full Precision)
base_model = AutoModelForCausalLM.from_pretrained(
"google/gemma-2-9b-it",
device_map="auto",
torch_dtype=torch.bfloat16,
)
model = PeftModel.from_pretrained(base_model, "dsfsi/gemma_2_9b_it-lora-r64-eng-yor")
Training Details
Hyperparameters
Table with columns: Parameter, Value| Parameter | Value |
|---|
| LoRA Rank (r) | 64 |
| LoRA Alpha | 128 |
| LoRA Dropout | 0.05 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Epochs | 3 |
| Batch Size | 2 |
| Learning Rate | 2e-04 |
| Max Sequence Length | 512 |
| Gradient Accumulation | 4 |
Hardware Requirements
Table with columns: Configuration, VRAM Required| Configuration | VRAM Required |
|---|
| 4-bit (QLoRA) | ~8-12 GB |
| 8-bit | ~16-20 GB |
| Full precision | ~24-40 GB |
Reproducibility
To reproduce this adapter:
# Clone the AfriScience-MT repository
git clone https://github.com/afriscience-mt/afriscience-mt.git
cd afriscience-mt
# Install dependencies
pip install -r requirements.txt
# Run LoRA training
python -m afriscience_mt.scripts.run_lora_training \
--data_dir ./data \
--source_lang eng \
--target_lang yor \
--model_name google/gemma-2-9b-it \
--model_type gemma \
--lora_rank 64 \
--output_dir ./output \
--num_epochs 3 \
--batch_size 4 \
--load_in_4bit
Limitations
- Domain Specificity: Optimized for scientific/academic texts; may underperform on casual or colloquial language.
- Language Direction: Only supports English → Yoruba translation.
- Base Model Required: Must be used with the google/gemma-2-9b-it base model.
- Context Length: Maximum context is model-dependent; longer texts should be chunked.
Citation
If you use this model, please cite the AfriScience-MT paper (arXiv:2605.29741):
@article{abdulmumin2026afriscience,
title = {AfriScience-MT: Towards Decolonizing Science in Africa through Text Translation},
author = {Abdulmumin, Idris and Gwadabe, Tajuddeen and Muhammad, Shamsuddeen Hassan and Adelani, David Ifeoluwa and Khalo, Nomonde and Ahmad, Ibrahim Said and Modupe, Abiodun and Mumm, Anina and Biyela, Sibusiso and Rabie, Michelle and Havemann, Johanna and Rei, Marek and Abbott, Jade and Marivate, Vukosi},
journal = {arXiv preprint arXiv:2605.29741},
year = {2026},
url = {https://arxiv.org/abs/2605.29741}
}
License
This adapter is released under the Apache 2.0 License.
Acknowledgments
