GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
roshangrewal
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
roshangrewal
Model Tree
google/gemma-4-E4B-it
Input Modalities
Output Modalities
Supported Functionality
python
from transformers import AutoProcessor, AutoModelForMultimodalLMfrom peft import PeftModelimport torch # Load base + adapterbase = AutoModelForMultimodalLM.from_pretrained( "google/gemma-4-E4B-it", torch_dtype=torch.bfloat16, device_map="auto",)model = PeftModel.from_pretrained(base, "roshangrewal/gemma4-e4b-toolcall-v02-lora")model.eval() processor = AutoProcessor.from_pretrained("google/gemma-4-E4B-it") # Define tools and querytools = [{"type": "function", "function": { "name": "get_weather", "description": "Get weather for a city", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]messages = [{"role": "user", "content": "What's the weather in Mumbai?"}] text = processor.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)inputs = processor(text=text, return_tensors="pt").to(model.device) output = model.generate(**inputs, max_new_tokens=200, do_sample=False)print(processor.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))# <|tool_call>call:get_weather{city:<|"|>Mumbai<|"|>}<tool_call|>| Parameter | Value |
|---|---|
| LoRA rank | 64 |
| LoRA alpha | 128 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Trainable params | 169M (2.08% of 8.1B) |
| Adapter size | ~679 MB |
| Training | 5000 steps, 78K examples, ~85 hours, Unsloth |
| Category | Accuracy |
|---|---|
| Multiple | 95.0% |
| Parallel | 90.0% |
| Simple Python | 88.5% |
| Parallel Multiple | 86.0% |
| Live Simple | 79.8% |
| Non-Live Average | 86.5% |
| Category | Accuracy |
|---|---|
| Simple | 100% |
| Complex Params | 100% |
| Many Tools (12+) | 93% |
| Ambiguous | 91.5% |
| No-Tool-Needed | 87.5% |
| OVERALL | 94.4% |
from transformers import AutoProcessor, AutoModelForMultimodalLMfrom peft import PeftModelimport torch # Load base + adapterbase = AutoModelForMultimodalLM.from_pretrained( "google/gemma-4-E4B-it", torch_dtype=torch.bfloat16, device_map="auto",)model = PeftModel.from_pretrained(base, "roshangrewal/gemma4-e4b-toolcall-v02-lora")model.eval() processor = AutoProcessor.from_pretrained("google/gemma-4-E4B-it") # Define tools and querytools = [{"type": "function", "function": { "name": "get_weather", "description": "Get weather for a city", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]messages = [{"role": "user", "content": "What's the weather in Mumbai?"}] text = processor.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)inputs = processor(text=text, return_tensors="pt").to(model.device) output = model.generate(**inputs, max_new_tokens=200, do_sample=False)print(processor.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))# <|tool_call>call:get_weather{city:<|"|>Mumbai<|"|>}<tool_call|>