GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
insraq
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
insraq
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Parameter | Value |
|---|---|
| direction_index | per layer |
| attn.o_proj.max_weight | 1.38 |
| attn.o_proj.max_weight_position | 35.21 |
| attn.o_proj.min_weight | 1.37 |
| attn.o_proj.min_weight_distance | 20.44 |
| mlp.down_proj.max_weight | 1.05 |
| mlp.down_proj.max_weight_position | 32.84 |
| mlp.down_proj.min_weight | 1.00 |
| mlp.down_proj.min_weight_distance | 15.18 |
| Metric | This model | Original model (ai9stars/G9v3-3B) |
|---|---|---|
| KL divergence | 0.0410 | 0 (by definition) |
| Refusals | 9/100 | 100/100 |
G9v3-3B is a dense 3B causal language model from the AI9Stars team, built for local deployment and resource-constrained scenarios. It targets everyday assistant use, coding, tool-use workflows, and reasoning tasks where a compact model is preferred.
LlamaForCausalLMbash
pip install "vllm>=0.21"vllm serve ai9stars/G9v3-3B --port 8000
bash
curl http://localhost:8000/v1/chat/completions \-H "Content-Type: application/json" \-d '{"model": "ai9stars/G9v3-3B","messages": [{"role": "user", "content": "Who are you?"}],"max_tokens": 128,"temperature": 0.7}'
bash
pip install "sglang[srt]>=0.5.12"python -m sglang.launch_server --model-path ai9stars/G9v3-3B --port 30000
bash
curl http://localhost:30000/v1/chat/completions \-H "Content-Type: application/json" \-d '{"model": "ai9stars/G9v3-3B","messages": [{"role": "user", "content": "Who are you?"}],"max_tokens": 128,"temperature": 0.7}'
bash
pip install -U "transformers>=5.6" accelerate torch
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "ai9stars/G9v3-3B"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id,torch_dtype="auto",device_map="auto",)messages = [{"role": "user", "content": "Who are you?"}]inputs = tokenizer.apply_chat_template(messages,tokenize=True,add_generation_prompt=True,enable_thinking=False,return_dict=True,return_tensors="pt",).to(model.device)outputs = model.generate(**inputs, max_new_tokens=128)print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Recommended sampling parameters:
| Mode | Recommended params | Enable |
|---|---|---|
| Think | temperature=0.9, top_p=0.95 | enable_thinking=True |
| No Think | temperature=0.7, top_p=0.95 | enable_thinking=False |
G9v3-3B is a language model that generates content based on learned statistical patterns from training data. It may produce inaccurate, biased, or unsafe outputs, and generated content should be reviewed and verified before use in high-stakes settings. Users are responsible for evaluating outputs, applying appropriate safeguards, and complying with applicable laws, regulations, and platform policies.
This repository and the G9v3 model weights are released under the Apache-2.0 License.