GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Jazhyc
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Jazhyc
Model Tree
Jazhyc/Gleipnir-4B
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Jazhyc/Gleipnir-4B-BF16"tokenizer = AutoTokenizer.from_pretrained(model_id, revision="v0.4")model = AutoModelForCausalLM.from_pretrained( model_id, revision="v0.4", torch_dtype=torch.bfloat16, device_map="auto").eval() instruction_path = hf_hub_download("Jazhyc/Gleipnir-4B", "student_prompt.txt")with open(instruction_path, encoding="utf-8") as handle: instruction = handle.read()trajectory = "User asked for a file listing. The agent ran: ls -la"user_prompt = ( f"{instruction}\n<agent_trajectory>\n{trajectory}\n" "</agent_trajectory>\n")prompt = tokenizer.apply_chat_template( [{"role": "user", "content": user_prompt}], tokenize=False, add_generation_prompt=True, enable_thinking=False,) + "Prediction:"inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device) token_ids = [ tokenizer.encode(label, add_special_tokens=False)[0] for label in ("0", "1")]with torch.no_grad(): next_logits = model(**inputs).logits[0, -1, token_ids].float()risk_score = torch.softmax(next_logits, dim=0)[1].item()print(risk_score)For throughput on any GPU, serve this repository with stock vLLM 0.31.
Its pooling runner turns the model into a classifier over the 0 and 1
logits at the last position:
bash
vllm serve Jazhyc/Gleipnir-4B-BF16 --revision v0.4 --served-model-name monitor \ --dtype bfloat16 --language-model-only --max-model-len 32768 --no-enable-prefix-caching \ --runner pooling --convert classify \ --pooler-config '{"pooling_type": "LAST", "use_activation": false}' \ --hf-overrides '{"classifier_from_token": ["0", "1"], "method": "no_post_processing", "num_labels": 2, "is_causal": true, "text_config": {"classifier_from_token": ["0", "1"], "method": "no_post_processing", "num_labels": 2, "is_causal": true, "mamba_ssm_dtype": "float32"}}' curl -s localhost:8000/classify -H 'Content-Type: application/json' -d '{"model": "monitor", "input": "<rendered prompt>", "add_special_tokens": false, "use_activation": false}'# data[0].probs holds the raw [logit_0, logit_1]; score = softmax(...)[1]The classifier settings repeat inside text_config because the Qwen
checkpoint wraps the language model. mamba_ssm_dtype keeps the GDN state
in FP32, as in the project's runs. Send the same rendered prompt as in the
Transformers example. On an RTX PRO 6000 this reproduced the project's
recorded BF16 results (ID 0.9335 against 0.9332, OOD 0.8045 against 0.8039
pAUROC@20; per-row MAE 0.004) at 36K prompt tokens/s.
For the highest throughput on an NVIDIA B200, use the project's optimized serving image, which loads this checkpoint and quantizes it at load time; see "Optimized B200 serving" in the main model card.
MIT, as for the adapter. Users must also comply with the Qwen/Qwen3.5-4B
terms and the upstream dataset terms described in the main model card.
import torchfrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Jazhyc/Gleipnir-4B-BF16"tokenizer = AutoTokenizer.from_pretrained(model_id, revision="v0.4")model = AutoModelForCausalLM.from_pretrained( model_id, revision="v0.4", torch_dtype=torch.bfloat16, device_map="auto").eval() instruction_path = hf_hub_download("Jazhyc/Gleipnir-4B", "student_prompt.txt")with open(instruction_path, encoding="utf-8") as handle: instruction = handle.read()trajectory = "User asked for a file listing. The agent ran: ls -la"user_prompt = ( f"{instruction}\n<agent_trajectory>\n{trajectory}\n" "</agent_trajectory>\n")prompt = tokenizer.apply_chat_template( [{"role": "user", "content": user_prompt}], tokenize=False, add_generation_prompt=True, enable_thinking=False,) + "Prediction:"inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device) token_ids = [ tokenizer.encode(label, add_special_tokens=False)[0] for label in ("0", "1")]with torch.no_grad(): next_logits = model(**inputs).logits[0, -1, token_ids].float()risk_score = torch.softmax(next_logits, dim=0)[1].item()print(risk_score)vllm serve Jazhyc/Gleipnir-4B-BF16 --revision v0.4 --served-model-name monitor \ --dtype bfloat16 --language-model-only --max-model-len 32768 --no-enable-prefix-caching \ --runner pooling --convert classify \ --pooler-config '{"pooling_type": "LAST", "use_activation": false}' \ --hf-overrides '{"classifier_from_token": ["0", "1"], "method": "no_post_processing", "num_labels": 2, "is_causal": true, "text_config": {"classifier_from_token": ["0", "1"], "method": "no_post_processing", "num_labels": 2, "is_causal": true, "mamba_ssm_dtype": "float32"}}' curl -s localhost:8000/classify -H 'Content-Type: application/json' -d '{"model": "monitor", "input": "<rendered prompt>", "add_special_tokens": false, "use_activation": false}'# data[0].probs holds the raw [logit_0, logit_1]; score = softmax(...)[1]