Model Details
Table with columns: Property, Value| Property | Value |
|---|
| Base model | mistralai/Mistral-7B-Instruct-v0.3 |
| Fine-tuning method | QLoRA (4-bit NF4 + LoRA adapters) |
| LoRA rank / alpha | r=16 / α=32 |
| Training epochs | 3 |
| Effective batch size | 16 (2 × 8 grad accumulation steps) |
| Learning rate | 2e-4 (cosine schedule, 3 % warmup) |
| Max sequence length | 2048 tokens |
| Training hardware | Kaggle T4 × 2 (via Unsloth) |
| Training framework | Unsloth + HuggingFace TRL SFTTrainer |
| Precision | FP16 |
| Language | French (fr) |
| License | MIT |
Training Dataset
The SFT dataset (togolm/togolm-corpus-v1)
consists of instruction–response pairs generated from the TogoLM corpus — a curated collection of
documents scraped from Togolese official sources:
Table with columns: Source, Domain| Source | Domain |
|---|
jo.gouv.tg | Journal Officiel — laws and decrees |
presidence.gouv.tg | Presidency — presidential acts and speeches |
assemblee-nationale.tg | National Assembly — parliamentary texts |
inseed.tg | National Statistics Institute — economic and demographic data |
service-public.gouv.tg | Public services directory |
| / / |
Q&A pairs were generated using Gemini 2.5 Flash and formatted in the Alpaca instruction template.
Usage
Load with Unsloth (recommended)
from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="togolm/togolm-7b-instruct-v1", max_seq_length=2048, load_in_4bit=True,)FastLanguageModel.for_inference(model) prompt = """Below is an instruction about Togo. Write a response that answers it accurately. ### Instruction:Quel est le taux d'imposition sur les sociétés au Togo ? ### Response:""" inputs = tokenizer(prompt, return_tensors="pt").to("cuda")outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)print(tokenizer.decode(outputs[0], skip_special_tokens=True))
from transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel base = AutoModelForCausalLM.from_pretrained( "mistralai/Mistral-7B-Instruct-v0.3", load_in_4bit=True, device_map="auto",)model = PeftModel.from_pretrained(base, "togolm/togolm-7b-instruct-v1")tokenizer = AutoTokenizer.from_pretrained("togolm/togolm-7b-instruct-v1")
The model was fine-tuned with the Alpaca instruction template:
Below is an instruction about Togo. Write a response that answers it accurately.### Instruction:{your question about Togo}### Response:
Intended Use
- Answering questions about Togolese law, administration, statistics, and public services in French
- Retrieval-augmented generation (RAG) combined with the TogoLM corpus
- Research on low-resource African languages and francophone AI
Out-of-Scope Use
- General-purpose chat or tasks unrelated to Togo
- Legal or medical advice — always verify with official Togolese sources
- Languages other than French (coverage is limited)
Project
This model is part of TogoLM — the first open-source AI infrastructure layer focused on Togo,
covering corpus collection, RAG engine, fine-tuned LLM, and a public REST API.
Citation
@misc{togolm2026, author = {Kougbada, Omar Farouk}, title = {TogoLM: Open-Source AI Infrastructure for Togo}, year = {2026}, howpublished = {\url{https://huggingface.co/togolm/togolm-7b-instruct-v1}},}