What This Is
An experimental research artifact exploring whether autonomous AI research systems can efficiently adapt large language models for low-resource African languages.
Despite 100M+ speakers, Hausa is severely underrepresented in language models. This adapter was produced by AutoScientist handling the entire pipeline: dataset selection (Dialectra Hausa Speech Corpus), preprocessing, training config, and execution.
Key point: This demonstrates AI-assisted language tech development for underrepresented communities.
Quick Start
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch
base_model = AutoModelForCausalLM.from_pretrained(
"mistralai/Mixtral-8x7B-Instruct-v0.1",
load_in_4bit=True,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mixtral-8x7B-Instruct-v0.1")
model = PeftModel.from_pretrained(base_model, "k0ns0l/mixtral-8x7b-hausa-lora-v1")
prompt = "Yaya kake?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Hardware: 24GB+ GPU (4-bit quantization) or 80GB (full precision)
Training Details
Generated by AutoScientist:
- Dataset: Dialectra Hausa Speech Corpus v1 (speech transcriptions)
- LoRA: r=64, alpha=128, targets: q/k/v/o_proj
- Training: 5 epochs, 1,115 steps, bf16
- Loss: 1.65 → 0.83
Why Mixtral: MoE efficiency, 32k context, multilingual foundation, instruction-following
Status & Limitations
⚠️ Experimental - qualitative testing shows good Hausa generation, but comprehensive benchmarking incomplete.
Limitations:
- Primarily Nigerian Hausa dialect
- Training data from speech transcriptions (conversational style)
- Not evaluated on FLORES-200, MasakhaNER, or other benchmarks yet
- Inherits base model biases
Use responsibly: Validate outputs with native speakers, don't use for high-stakes decisions without oversight.
About AutoScientist
AutoScientist by AdaptionLabs automates ML research workflows—from experiment design to execution. Reported 35% improvement over human-configured pipelines.
This project demonstrates autonomous research for low-resource language adaptation.
Citation
@misc{k0ns0l2026mixtral-hausa,
author = {k0ns0l},
title = {Mixtral-8x7B Hausa LoRA v1: AutoScientist-Generated Adapter},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/k0ns0l/mixtral-8x7b-hausa-lora-v1}}
}
Contact: @k0ns0l | License: Apache 2.0 | Contributions welcome
Research artifact exploring autonomous AI for underrepresented language technology.