GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ChatoyantAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ChatoyantAI
Model Tree
google/gemma-4-12B-it
Input Modalities
Output Modalities
Supported Functionality
google/gemma-4-12B-it base.| Item | Value |
|---|---|
| Architecture | Gemma4UnifiedForConditionalGeneration |
| Global step / completed epoch | 6126 / 2.0 |
| Adaptation | LoRA rank 32, alpha 64, dropout 0 |
| Target modules | Language-model attention and MLP projections |
| Training framework | Unsloth with PEFT |
| Training data | Internal filtered roleplay conversations; not included in this release |
| Supervised target | Final target assistant response; earlier conversation is context |
| Final logged validation loss | 1.06754386 |
Validation loss is training-run telemetry, not a public benchmark or a claim of human preference improvement. No external benchmark accuracy is claimed. This release targets text conversations. Multimodal modules are retained from the base architecture, but multimodal quality has not been evaluated after SFT. Training data may include mature themes; outputs require appropriate review. The model may generate inaccurate, biased, inappropriate, or memorized content.
The checkpoint tokenizer and chat template are included. Preserve the order
system → assistant greeting → user → assistant history … → user as applicable.
Do not replace this with another Gemma generation's template.
The deployed adapter service is configured for 65,536 total input plus output tokens.
The base architectural context setting is preserved; long-context quality after
fine-tuning has not been independently benchmarked.
Requires a Transformers release with Gemma4Unified support. Export environment: Transformers 5.10.1, PEFT 0.18.1, PyTorch 2.11.0+cu130. Use suitable CUDA wheels for your machine.
python
import torchfrom transformers import Gemma4UnifiedForConditionalGeneration, AutoTokenizermodel_id = "ChatoyantAI/gemma-4-12b-it-roleplay-sft-epoch2-lora"tokenizer = AutoTokenizer.from_pretrained(model_id)model = Gemma4UnifiedForConditionalGeneration.from_pretrained('google/gemma-4-12B-it', dtype=torch.bfloat16, device_map="auto")from peft import PeftModelmodel = PeftModel.from_pretrained(model, model_id)messages = [{"role": "system", "content": "You are a friendly cafe host. Reply in English."},{"role": "assistant", "content": "Welcome! Would you like a table by the window?"},{"role": "user", "content": "Yes, please. Could I have some tea?"},]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,return_dict=True, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=256)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
For the standalone BF16 variant, an example vLLM launch is:
bash
vllm serve ChatoyantAI/gemma-4-12b-it-roleplay-sft-epoch2-bf16 --dtype bfloat16 --max-model-len 65536 --served-model-name roleplay
This command requires sufficient GPU memory and a vLLM version supporting the architecture. A 64K configuration is not a guarantee of any particular concurrency. No FP8/INT4 quantization is applied to this release.
release-manifest.json records source weight hashes, adapter configuration,
export versions, merge validation and file hashes. Training examples, optimizer
states, raw run logs and credentials are deliberately excluded.
Apache License 2.0, following the original base model license. See LICENSE,
NOTICE, and BASE_MODEL_README.md for the original model attribution.
The model weights were modified by merging the roleplay LoRA (standalone variant)
or are provided as the corresponding separately trained adapter (LoRA variant).
This derivative is not an official Google release.