GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
iulio
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
iulio
Model Tree
google/gemma-4-26B-A4B-it
Input Modalities
Output Modalities
Supported Functionality
| Attribute | Specification |
|---|---|
| Base Model | google/gemma-4-26B-A4B-it (Google DeepMind) |
| Architecture | Sparse Mixture-of-Experts (gemma4_unified) with 128 experts, top-8 routing |
| Active Parameters | ~4 Billion active parameters per token (25.82B total parameters) |
| Hardware Used | NVIDIA A100-SXM4-80GB (Modal.com Serverless Cluster) |
| Quantization at Training | 4-bit NF4 with native bfloat16 compute (BitsAndBytesConfig) |
| LoRA Config | r=16, α=32, targeting language model linear projections (q, k, v, o, gate, up, down) |
| Training Steps | 100 steps (effective batch size 8) over 1,719 statutory curriculum items |
| Training Duration | 979.9 seconds (16.33 min) |
| Final Loss | 0.6011 (Mean token accuracy >99.9%) |
| Adapter Size | 35.51 MB SafeTensors |
python
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel BASE_MODEL = "google/gemma-4-26B-A4B-it"ADAPTER_REPO = "iulio/FiscMind-Gemma4-26B-MoE" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, device_map="auto", dtype=torch.bfloat16,) model = PeftModel.from_pretrained(base_model, ADAPTER_REPO)model.eval() sys_msg = ( "Esti FiscMind AI, expert contabil CECCAR si consultant fiscal CCF conform " "legislatiei din Romania actualizate 2025/2026. Raspunde precis cu temei legal si formule contabile complete.")prompt = f"""<start_of_turn>user{sys_msg} Care este monografia contabila pentru inregistrarea dividendelor interimare trimestriale si ce impozit se retine?<end_of_turn><start_of_turn>model""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=512, temperature=0.2, do_sample=True, ) print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))Developed by Iulian Amaricai as part of the FiscMind AI Romanian Statutory Accounting Assistant initiative. Base model courtesy of Google DeepMind (Gemma 4 Family).