GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nam194
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
nam194
Model Tree
Qwen/Qwen3.5-4B-Base
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration MODEL = "nam194/cribl-decision-1.0-merged" tokenizer = AutoTokenizer.from_pretrained(MODEL)model = Qwen3_5ForConditionalGeneration.from_pretrained( MODEL, dtype=torch.bfloat16).to("cuda").eval()Important: Use
Qwen3_5ForConditionalGeneration, notAutoModelForCausalLM, which can silently ignore the merged weights for this architecture.
The model does not generate text. Run one forward pass and restrict the
lm_head to the candidate-letter token IDs:
python
import json, torch LETTERS = "ABCDEFGHIJ"base_model = model # no PEFT wrapper neededlanguage_model = base_model.model.language_modellm_head = base_model.lm_head @torch.inference_mode()def score(prompt, n_options): ids = tokenizer.encode(prompt, add_special_tokens=False) input_ids = torch.tensor([ids], dtype=torch.long, device="cuda") mask = torch.ones_like(input_ids) hidden = language_model(input_ids=input_ids, attention_mask=mask).last_hidden_state h = hidden[0, -1] # hidden state at the "(" token letter_ids = [tokenizer.encode(LETTERS[i], add_special_tokens=False)[0] for i in range(n_options)] cand_ids = torch.tensor(letter_ids, dtype=torch.long, device="cuda") weights = lm_head.weight[cand_ids] logits = (weights @ h).float() return torch.softmax(logits, dim=-1).cpu().tolist() state = {"message": "My card was charged twice for the same purchase."}question = "Which department should handle this?"options = ["billing", "technical support", "sales"]prompt = ( f"Context:\n{json.dumps(state)}\n\n" f"Question: {question}\n" f"Options:\n" + "\n".join(f"({LETTERS[i]}) {opt}" for i, opt in enumerate(options)) + "\nAnswer: (") probs = score(prompt, len(options))for i, (opt, p) in enumerate(zip(options, probs)): print(f"({LETTERS[i]}) {opt:20s} {p:.4f}")Prompt format, decision types, label table, and limitations are documented in the original adapter card. Using a different prompt format will degrade accuracy.
Apache-2.0, inherited from the base model and adapter.
import torchfrom transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration MODEL = "nam194/cribl-decision-1.0-merged" tokenizer = AutoTokenizer.from_pretrained(MODEL)model = Qwen3_5ForConditionalGeneration.from_pretrained( MODEL, dtype=torch.bfloat16).to("cuda").eval()import json, torch LETTERS = "ABCDEFGHIJ"base_model = model # no PEFT wrapper neededlanguage_model = base_model.model.language_modellm_head = base_model.lm_head @torch.inference_mode()def score(prompt, n_options): ids = tokenizer.encode(prompt, add_special_tokens=False) input_ids = torch.tensor([ids], dtype=torch.long, device="cuda") mask = torch.ones_like(input_ids) hidden = language_model(input_ids=input_ids, attention_mask=mask).last_hidden_state h = hidden[0, -1] # hidden state at the "(" token letter_ids = [tokenizer.encode(LETTERS[i], add_special_tokens=False)[0] for i in range(n_options)] cand_ids = torch.tensor(letter_ids, dtype=torch.long, device="cuda") weights = lm_head.weight[cand_ids] logits = (weights @ h).float() return torch.softmax(logits, dim=-1).cpu().tolist() state = {"message": "My card was charged twice for the same purchase."}question = "Which department should handle this?"options = ["billing", "technical support", "sales"]prompt = ( f"Context:\n{json.dumps(state)}\n\n" f"Question: {question}\n" f"Options:\n" + "\n".join(f"({LETTERS[i]}) {opt}" for i, opt in enumerate(options)) + "\nAnswer: (") probs = score(prompt, len(options))for i, (opt, p) in enumerate(zip(options, probs)): print(f"({LETTERS[i]}) {opt:20s} {p:.4f}")