GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
gmrrh
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
gmrrh
Model Tree
ibm-granite/granite-4.2-8b
Input Modalities
Output Modalities
Supported Functionality
| Architecture | GraniteForCausalLM (inherited from base) |
| Parameters | ~8.8B, bf16 |
| Layers / hidden size | 40 / 4096 |
| Attention | 32 heads, 8 KV heads (GQA) |
| Context | 131,072 tokens (native) |
| Vocabulary | 100,352 (GPT-2-style BPE) |
| Fine-tune | LoRA r=64, α=128, dropout 0.05, all linear projections |
Kiesel is designed as a conversational assistant for everyday consumer tasks — appointments, reminders, lookups, small transactions — with native function calling and thinking-style reasoning, following the official Granite 4.2 chat template that ships with this repository (chat_template.jinja).
tools argument.Out of scope: This model is not a general-purpose knowledge model and is not intended for safety-critical decisions, medical/legal/financial advice, or use cases requiring guarantees of factual accuracy. It inherits the base model's limitations.
Fine-tuned on 7,609 unique German tool-calling conversations spanning four bands: core assistant scenarios, edge cases, premise/logic questions, and a robustness band (refusals, prompt-injection resistance, overblocking guards, persona consistency, noise tolerance). A portion of the data is synthetic, produced and quality-filtered with LLM-based generation and judging.
Training configuration: 3 epochs, learning rate 1e-4 (cosine schedule), sequence length 6,144, LoRA r=64/α=128 on all linear projections, bf16, trained with Levanter on a TPU v6e-4 pod. The adapter was then merged into the base weights; the standalone adapter is published separately.
The production system prompt and the full production tool registry are not part of this release; the examples below use illustrative placeholders.
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "gmrrh/kiesel-v2"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto") messages = [ { "role": "system", "content": "Du bist ein freundlicher Assistent. (Beispiel-Prompt — nicht der Produktions-Prompt)", }, {"role": "user", "content": "Wie viele Tage hat ein Schaltjahr?"},] text = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)inputs = tok(text, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=1024, do_sample=False)print(tok.decode(output[0][inputs.shape[1]:], skip_special_tokens=True))Tools are passed in OpenAI function-schema style. The model answers with a
<tool_call><function=name><parameter=...>...</parameter></function></tool_call>
block that you parse and execute, then continue the conversation with a
role: "tool" message.
python
tools = [ { "type": "function", "function": { "name": "create_appointment", "description": "Legt einen Kalendereintrag an.", "parameters": { "type": "object", "properties": { "title": {"type": "string", "description": "Titel des Termins"}, "date": {"type": "string", "description": "Datum im Format YYYY-MM-DD"}, "time": {"type": "string", "description": "Uhrzeit im Format HH:MM"}, }, "required": ["title", "date"], }, }, }, { "type": "function", "function": { "name": "get_weather", "description": "Aktuelles Wetter für eine Stadt.", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, },] messages = [ {"role": "system", "content": "Du bist ein Assistent mit Kalenderzugriff. (Beispiel-Prompt)"}, {"role": "user", "content": "Bitte lege für nächsten Montag um 9 Uhr einen Zahnarzttermin an."},] text = tok.apply_chat_template(messages, tools=tools, add_generation_prompt=True, tokenize=False)The chat template supports thinking-style reasoning (<think>...</think> before the
final answer, enabled by default). You can control it via apply_chat_template
keyword arguments:
enable_thinking=False — skip reasoning entirely for faster answersreasoning_effort="low" — a shorter reasoning budgettruncate_history_thinking=False — keep <think> blocks from earlier turns
instead of stripping them from the conversation historybash
vllm serve gmrrh/kiesel-v2 \ --served-model-name kiesel-v2 \ --max-model-len 65536 \ --gpu-memory-utilization 0.90 \ --tool-call-parser qwen3_coder \ --reasoning-parser granite_thinking_parsergranite_thinking_parser is the reasoning parser shipped with the upstream
Granite 4.2 release and matches the <think> format this model was trained with.
Structured evaluation is in progress and will be published in this README as it completes. Planned categories: function/tool calling, instruction following, safety and refusals, math, long-context retrieval, German and English knowledge, and agentic tasks.
This model is released under the Apache License 2.0, consistent with its base model ibm-granite/granite-4.2-8b by IBM. Kiesel is an independent fine-tune and is not an IBM product.
Kiesel · gmrrh · D. Gutz
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "gmrrh/kiesel-v2"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto") messages = [ { "role": "system", "content": "Du bist ein freundlicher Assistent. (Beispiel-Prompt — nicht der Produktions-Prompt)", }, {"role": "user", "content": "Wie viele Tage hat ein Schaltjahr?"},] text = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)inputs = tok(text, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=1024, do_sample=False)print(tok.decode(output[0][inputs.shape[1]:], skip_special_tokens=True))tools = [ { "type": "function", "function": { "name": "create_appointment", "description": "Legt einen Kalendereintrag an.", "parameters": { "type": "object", "properties": { "title": {"type": "string", "description": "Titel des Termins"}, "date": {"type": "string", "description": "Datum im Format YYYY-MM-DD"}, "time": {"type": "string", "description": "Uhrzeit im Format HH:MM"}, }, "required": ["title", "date"], }, }, }, { "type": "function", "function": { "name": "get_weather", "description": "Aktuelles Wetter für eine Stadt.", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, },] messages = [ {"role": "system", "content": "Du bist ein Assistent mit Kalenderzugriff. (Beispiel-Prompt)"}, {"role": "user", "content": "Bitte lege für nächsten Montag um 9 Uhr einen Zahnarzttermin an."},] text = tok.apply_chat_template(messages, tools=tools, add_generation_prompt=True, tokenize=False)vllm serve gmrrh/kiesel-v2 \ --served-model-name kiesel-v2 \ --max-model-len 65536 \ --gpu-memory-utilization 0.90 \ --tool-call-parser qwen3_coder \ --reasoning-parser granite_thinking_parser