GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
spkc83
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
spkc83
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Qwen2MoeForCausalLMThe model is not a 9B model trained from random initialization. Compatible language, attention, embedding, and normalization weights were copied from the pinned 1.5B base checkpoint. The MoE routed branches were initialized to preserve the dense model's initial output, then trained for 1,000 optimizer steps.
Generative adaptation used:
The generative splits contain CDLA-Sharing-1.0 and MIT material. Banking77 (CC-BY-4.0) is reserved for intent-router evaluation and was not used for generative training.
These training metrics do not establish production quality. Broader held-out response, multi-turn, hallucination, safety, and calibrated domain-router evaluations remain required.
A deterministic six-scenario smoke test on an RTX PRO 6000 found that the model produced fluent banking responses and carried a declined-card scenario into a second turn. It also found release-blocking behavior:
The raw result is stored in
evals/smoke-20260727T122519Z.json. Do not expose the checkpoint without
external domain and sensitive-data controls.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "spkc83/retail-bank-servicing-moe-9b"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id,dtype=torch.bfloat16,device_map={"": 0},)# Router logits and auxiliary loss are training diagnostics. Disable them for# generation with the current Transformers Qwen2-MoE implementation.model.config.output_router_logits = Falsemodel.eval()messages = [{"role": "system","content": ("You are a retail banking support assistant. Help with accounts, cards, ""transfers, payments, loans, fees, branches, ATMs, and related ""financial-services support. If the user asks about another domain, ""give the standard out-of-domain response."),},{"role": "user", "content": "My debit card was stolen. What should I do?"},]inputs = tokenizer.apply_chat_template(messages,add_generation_prompt=True,return_tensors="pt",).to(model.device)with torch.inference_mode():output = model.generate(inputs, max_new_tokens=160, do_sample=False)print(tokenizer.decode(output[0, inputs.shape[-1] :], skip_special_tokens=True))
The public application uses a separate calibrated domain/intent router. An out-of-domain decision bypasses neural generation and returns:
I can only help with retail banking and financial-services questions. Please ask about accounts, cards, transfers, payments, loans, or related banking support.
Prompting or fine-tuning alone does not guarantee that exact response. The
released DistilBERT router has a binary supported-banking/OOD head and a 77-way
Banking77 intent head. Its held-out intent macro F1 is 0.951208, with OOD
false-accept rate 0.007733 at threshold 0.98. It still requires deterministic
credential and unsafe-output guards and is not production-qualified.
The public Retail Bank Servicing POC authenticates two static demo users, runs the learned CPU router, and executes this checkpoint on ZeroGPU for constrained tool selection and grounded response generation. Synthetic backend reads and writes remain behind deterministic identity, argument, authorization, and unsafe-output checks.
The deployment uses eager expert execution for compatibility with the current RTX PRO 6000 Blackwell partition. The raw checkpoint remains unsafe to expose without the application's external controls.
Use the model only for experimentation with human review.