Model details
Table | |
|---|
| Base model | unsloth/gemma-4-E2B-it (Unsloth's release of Google's Gemma 4 E2B-it) |
| Model size | 5.1B stored parameters (about 2B effective; Gemma 4 E2B architecture) |
| Language | Tetun (tdt) |
| Fine-tuning method | QLoRA (LoRA rank 16 on the 4-bit base model); the adapters were merged into the bfloat16 base weights |
| Context length | Fine-tuned on conversations of up to 4,096 tokens (the architecture supports 128K) |
| Other formats | GGUF, Q4_K_M · Ollama |
| License | Apache 2.0, following the Gemma 4 license |
Only the language model was fine-tuned. The vision and audio encoders are unchanged from the base model, and image or audio input in Tetun has not been trained or tested.
How to use
Requires transformers 5.5.0 or later.
pip install -U "transformers>=5.5.0" accelerate
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "labadain/Labadain-Gemma-4-E2B-it-Tetun" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, dtype=torch.bfloat16, device_map="auto",) messages = [ {"role": "user", "content": "Saida mak intelijénsia artifisiál?"},] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt", return_dict=True, enable_thinking=False, # the model was not trained to reason before answering).to(model.device) outputs = model.generate(**inputs, max_new_tokens=2048) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
The default sampling settings come from generation_config.json (temperature 1.0, top-p 0.95, top-k 64). For repeatable answers, pass do_sample=False.
Load in 4-bit (less GPU memory)
The bfloat16 weights need about 10 GB of GPU memory. Loading in 4-bit uses much less, at a small cost in quality.
With Transformers (requires pip install bitsandbytes): load the model like this, then generate exactly as in the example above.
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig model_id = "labadain/Labadain-Gemma-4-E2B-it-Tetun" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, quantization_config=BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, ), device_map="auto",)
With Unsloth (requires pip install unsloth):
from unsloth import FastModel model, processor = FastModel.from_pretrained( model_name="labadain/Labadain-Gemma-4-E2B-it-Tetun", max_seq_length=4096, load_in_4bit=True, # set to False to load the full bfloat16 weights)FastModel.for_inference(model) messages = [ { "role": "user", "content": [{"type": "text", "text": "Saida mak intelijénsia artifisiál?"}], },] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=False,).to(model.device) outputs = model.generate(**inputs, max_new_tokens=2048) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Tips
- Keep thinking mode off. The training conversations contain no reasoning traces, so
enable_thinking=True asks for behavior the model was not tuned for. The chat template already has thinking off by default.
- A system prompt is optional. The model was fine-tuned without one. A short instruction such as "Responde iha Tetun" can still help keep answers in Tetun.
- Multi-turn chat works. About a third of the training conversations have two or more question–answer exchanges. Pass the full history in
messages.
Training data
The model was fine-tuned on 32,197 Tetun conversations (about 26 million tokens) compiled by Labadain from several Tetun conversational and question–answer resources, including some material machine-translated into Tetun.
Table with columns: Domain, Conversations| Domain | Conversations |
|---|
| News articles | 1,951 |
| Health | 9,649 |
| Education and language | 465 |
| General conversations | 20,132 |
| Total | 32,197 |
Together the conversations contain 55,105 question–answer exchanges. 11,774 conversations (36.6%) are multi-turn, with up to 33 exchanges, and 20,423 (63.4%) are single-turn. 90% of the conversations (28,977) were used for training, and the remaining 10% were split evenly into a validation set (1,610) and a held-out test set (1,610).
Intended uses
- Tetun text generation and conversational assistants
- Research on multilingual and low-resource NLP
- Education, and prototyping Tetun-language AI applications for Timor-Leste
Limitations
- Factual errors. This is a small model and can state incorrect facts, dates, names and figures with confidence, including about Timor-Leste. Check important information against reliable sources.
- Quantization. The 4-bit GGUF can be slightly less accurate than the original bfloat16 model. Use labadain/labadain-E2B-it-Tetun for the best quality.
- Fixed knowledge. The model only knows what was in the base model and the training data. It has no access to current events.
- Long inputs. Fine-tuning covered conversations of up to 4,096 tokens. Quality on longer conversations or documents has not been tested.
- No benchmark evaluation. The model has not been evaluated on standard benchmarks beyond what the base model provides.
License
This model is released under the Apache 2.0 license, following the Gemma 4 license of the base model.
Acknowledgements
Built on Gemma 4 by Google DeepMind and fine-tuned with Unsloth.
About Labadain
Labadain is a research initiative advancing information retrieval (IR), natural language processing (NLP), and artificial intelligence (AI) for Tetun and other low-resource language settings, with a focus on promoting digital inclusion and AI access in Timor-Leste.
Citation
@misc{labadain2026gemma4e2btetun, title = {Labadain-Gemma-4-E2B-it-Tetun}, author = {Gabriel de Jesus}, year = {2026}, howpublished = {\url{https://huggingface.co/labadain/Labadain-Gemma-4-E2B-it-Tetun}}}