GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
saliltambe
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
saliltambe
Model Tree
google/gemma-4-E2B-it
Input Modalities
Output Modalities
Supported Functionality
The base model already produces fluent Nepali when given the system prompt "You are a helpful AI assistant that answers in Nepali.", and English without it. This adapter distills that system prompt into the weights, so the behaviour is unconditional. The point is on-device efficiency: a system prompt costs prefill tokens on every request, and training it in once removes that cost permanently.
OpenAssistant/oasst1, English conversation openers (Apache-2.0, human-written)google/gemma-4-E2B-it itself with the system prompt aboveOn 60 held-out English prompts, with no system prompt, mean fraction of Devanagari characters in the response:
| Devanagari | |
|---|---|
base google/gemma-4-E2B-it | 0.000 |
| base + this adapter | 0.814 |
| Setting | Value |
|---|---|
| rank / alpha / dropout | 16 / 32 / 0.05 |
| target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (205 modules, text decoder only) |
| epochs | 3 |
| effective batch size | 16 |
| learning rate | 0.0002 |
| max sequence length | 768 |
| precision | bf16 base, fp32 LoRA |
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("google/gemma-4-E2B-it", dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "saliltambe/gemma-4-E2B-it-nepali-lora")tok = AutoTokenizer.from_pretrained("google/gemma-4-E2B-it")
For on-device deployment with LiteRT-LM, see the companion conversion notebook.