GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
ghananlpcommunity
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ghananlpcommunity
Model Tree
openbmb/MiniCPM5-1B
Input Modalities
Output Modalities
Supported Functionality
Use sampling with a repetition penalty. Under greedy decoding this model degenerates into repetition loops. Recommended:
markdown
temperature=0.7, top_p=0.9, repetition_penalty=1.3, no_repeat_ngram_size=3
These are set as defaults in generation_config.json.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizermid = "ghananlpcommunity/MiniCPM5-1B-Twi"tok = AutoTokenizer.from_pretrained(mid)model = AutoModelForCausalLM.from_pretrained(mid, torch_dtype=torch.bfloat16, device_map="auto")msgs = [{"role": "user", "content": "Kyerɛ me nkyerɛkyerɛmu tiawa fa Ghana aduane ho."}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True,enable_thinking=False, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=200, do_sample=True,temperature=0.7, top_p=0.9, repetition_penalty=1.3, no_repeat_ngram_size=3)print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
1. Tokenizer extension. The base tokenizer split Twi very inefficiently — the Ghanaian vowels ɛ (U+025B) and ɔ (U+0254) each became two byte-level tokens, giving a Twi fertility of 2.90 tokens/word vs 1.29 for English. We trained an auxiliary byte-level BPE on clean Twi and added the top 6,000 Twi word-pieces to the tokenizer (130,560 → 136,560), then resized the model embeddings and warm-started each new token from the mean of its original subword embeddings. This cut Twi fertility to ~1.26 tokens/word (−57%).
2. Continued pretraining (CPT). Full-parameter CPT for 1 epoch (~666M tokens) on a mix of 65% monolingual Twi / 20% EN–Twi parallel / 15% English replay (the English replay mitigates catastrophic forgetting at this scale). Loss 6.29 → 1.90.
3. Supervised fine-tuning (SFT). LoRA SFT (rank 32) for 3 epochs on ~200k shuffled instruction/QA examples — a blend of short factual QA and longer passage-grounded QA in Twi. Loss 4.14 → 2.30.
All Twi text was Unicode-normalized to canonical Latin ɛ/ɔ before training.
| Stage | Dataset |
|---|---|
| Tokenizer + CPT (monolingual) | ghananlpcommunity/pristine-twi |
| CPT (parallel / grounding) | ghananlpcommunity/pristine-twi-english |
| SFT (short QA) | ghananlpcommunity/gooaq-twi-2m |
| SFT (long / grounded QA) | michsethowusu/ghana-chat-corpus-ak |
Held-out Twi perplexity after CPT: 8.26. Qualitatively the model produces fluent, idiomatic Twi with correct orthography. Descriptive/explanatory prompts work well.
This is a v1 focused on fluency, and it has real weaknesses:
Planned v2 improvements: cleaner/curated SFT data, instruction-format English replay during CPT, and chain-of-thought-shaped examples (reason in English → answer in Twi).
Built on openbmb/MiniCPM5-1B (Apache-2.0) using datasets from the Ghana NLP community. Released under Apache-2.0. Developed by michsethowusu / Ghana NLP.