GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Rumiii
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Rumiii
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
| Base model | Qwen3-8B |
| Method | QLoRA (4-bit), rank 16, alpha 16, all attention and MLP projections; merged to 16-bit |
| Data | 5,000 samples from Toucan-1.5M (SFT split): 2,000 multi-turn, 1,250 single-turn (original), 750 single-turn (diversified), 1,000 irrelevant-tool (no call expected) |
| Sequence length | Up to 4,096 tokens |
| Schedule | 1 epoch (625 steps), learning rate 2e-4 (cosine), effective batch size 8 |
| Loss | Assistant responses only |
| Hardware | 1 × NVIDIA T4, Unsloth |
The 16-bit weights are about 15 GB. On smaller GPUs, load the model in 4-bit (for example with bitsandbytes).
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "Rumiii/Qwen3-8B-Maxima_A1"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto") tools = [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, },}] messages = [{"role": "user", "content": "What's the weather in Hyderabad?"}]prompt = tok.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)inputs = tok(prompt, return_tensors="pt").to(model.device) out = model.generate(**inputs, max_new_tokens=256, do_sample=False)print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=False).split("<|im_end|>")[0])Expected output:
text
<tool_call>{"name": "get_weather", "arguments": {"city": "Hyderabad"}}</tool_call>To continue after running the tool, append the call and its result to messages, rebuild the prompt with apply_chat_template, and generate again:
python
messages += [ {"role": "assistant", "content": "", "tool_calls": [ {"type": "function", "function": {"name": "get_weather", "arguments": {"city": "Hyderabad"}}}]}, {"role": "tool", "content": '{"temp_c": 31, "condition": "Partly cloudy"}'},]Data: Toucan-1.5M (Apache 2.0). Base model: Qwen3-8B (Apache 2.0).
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "Rumiii/Qwen3-8B-Maxima_A1"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto") tools = [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, },}] messages = [{"role": "user", "content": "What's the weather in Hyderabad?"}]prompt = tok.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)inputs = tok(prompt, return_tensors="pt").to(model.device) out = model.generate(**inputs, max_new_tokens=256, do_sample=False)print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=False).split("<|im_end|>")[0])messages += [ {"role": "assistant", "content": "", "tool_calls": [ {"type": "function", "function": {"name": "get_weather", "arguments": {"city": "Hyderabad"}}}]}, {"role": "tool", "content": '{"temp_c": 31, "condition": "Partly cloudy"}'},]