GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ā
zenlm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
zenlm
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Zen Guard Stream is a 4B parameter token-level streaming classifier that evaluates each generated token in real time. It's optimized for real-time content moderation during LLM response generation.
text
User Prompt ā Zen Guard Stream (safety check) ā If Safe, continue āLLM Response ā Token by Token ā Real-time Classification ā Immediate intervention if unsafe| Specification | Value |
|---|---|
| Parameters | 4B |
| Type | Streaming Classifier |
| Base Model | Zen Guard |
| Context Length | 32,768 tokens |
| Languages | 119 |
| Latency | 5ms/token |
| VRAM (FP16) | 8GB |
| VRAM (INT8) | 4GB |
python
import torchfrom transformers import AutoModel, AutoTokenizerĀ model_path = "zenlm/zen-guard-stream"Ā tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)model = AutoModel.from_pretrained( model_path, device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True,).eval()Ā # Simulate streaming moderationuser_message = "Hello, how are you?"assistant_message = "I'm doing well, thank you for asking!"messages = [ {"role": "user", "content": user_message}, {"role": "assistant", "content": assistant_message}]Ā text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)inputs = tokenizer(text, return_tensors="pt")token_ids = inputs.input_ids[0]Ā # Initialize stream statestream_state = NoneĀ # Process tokens one by one (simulating streaming)for i, token_id in enumerate(token_ids): result, stream_state = model.stream_moderate_from_ids( token_id, role="assistant" if i > len(user_message) else "user", stream_state=stream_state ) token_str = tokenizer.decode([token_id]) risk = result['risk_level'][-1] print(f"Token: {repr(token_str)} ā Risk: {risk}")Ā model.close_stream(stream_state)python
from transformers import AutoModel, AutoTokenizer, TextIteratorStreamerimport threadingĀ # Load both modelsllm = AutoModelForCausalLM.from_pretrained("zenlm/zen-next")guard = AutoModel.from_pretrained("zenlm/zen-guard-stream", trust_remote_code=True)tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-next")Ā def safe_generate(prompt): # Initial prompt check result, state = guard.stream_moderate_from_ids( tokenizer.encode(prompt, return_tensors="pt")[0], role="user", stream_state=None ) if result['risk_level'][-1] != "Safe": return "I cannot process this request." # Stream generation with real-time moderation streamer = TextIteratorStreamer(tokenizer) inputs = tokenizer(prompt, return_tensors="pt") thread = threading.Thread(target=llm.generate, kwargs={**inputs, "streamer": streamer}) thread.start() output = [] for token in streamer: token_id = tokenizer.encode(token, add_special_tokens=False) if token_id: result, state = guard.stream_moderate_from_ids( token_id[0], role="assistant", stream_state=state ) if result['risk_level'][-1] == "Unsafe": thread.join() return "".join(output) + " [Content moderated]" output.append(token) guard.close_stream(state) return "".join(output)| Metric | Zen Guard Stream |
|---|---|
| Accuracy | 95.2% |
| F1 Score | 93.1% |
| False Positive | 2.8% |
| Latency | 5ms/token |
Apache 2.0
bibtex
@misc{zenguardstream2025, title={Zen Guard Stream: Real-Time Token-Level Safety Moderation}, author={Hanzo AI and Zoo Labs Foundation}, year={2025}, publisher={HuggingFace}, howpublished={\url{https://huggingface.co/zenlm/zen-guard-stream}}}tags:
User Prompt ā Zen Guard Stream (safety check) ā If Safe, continue āLLM Response ā Token by Token ā Real-time Classification ā Immediate intervention if unsafeimport torchfrom transformers import AutoModel, AutoTokenizerĀ model_path = "zenlm/zen-guard-stream"Ā tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)model = AutoModel.from_pretrained( model_path, device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True,).eval()Ā # Simulate streaming moderationuser_message = "Hello, how are you?"assistant_message = "I'm doing well, thank you for asking!"messages = [ {"role": "user", "content": user_message}, {"role": "assistant", "content": assistant_message}]Ā text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)inputs = tokenizer(text, return_tensors="pt")token_ids = inputs.input_ids[0]Ā # Initialize stream statestream_state = NoneĀ # Process tokens one by one (simulating streaming)for i, token_id in enumerate(token_ids): result, stream_state = model.stream_moderate_from_ids( token_id, role="assistant" if i > len(user_message) else "user", stream_state=stream_state ) token_str = tokenizer.decode([token_id]) risk = result['risk_level'][-1] print(f"Token: {repr(token_str)} ā Risk: {risk}")Ā model.close_stream(stream_state)from transformers import AutoModel, AutoTokenizer, TextIteratorStreamerimport threadingĀ # Load both modelsllm = AutoModelForCausalLM.from_pretrained("zenlm/zen-next")guard = AutoModel.from_pretrained("zenlm/zen-guard-stream", trust_remote_code=True)tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-next")Ā def safe_generate(prompt): # Initial prompt check result, state = guard.stream_moderate_from_ids( tokenizer.encode(prompt, return_tensors="pt")[0], role="user", stream_state=None ) if result['risk_level'][-1] != "Safe": return "I cannot process this request." # Stream generation with real-time moderation streamer = TextIteratorStreamer(tokenizer) inputs = tokenizer(prompt, return_tensors="pt") thread = threading.Thread(target=llm.generate, kwargs={**inputs, "streamer": streamer}) thread.start() output = [] for token in streamer: token_id = tokenizer.encode(token, add_special_tokens=False) if token_id: result, state = guard.stream_moderate_from_ids( token_id[0], role="assistant", stream_state=state ) if result['risk_level'][-1] == "Unsafe": thread.join() return "".join(output) + " [Content moderated]" output.append(token) guard.close_stream(state) return "".join(output)@misc{zenguardstream2025, title={Zen Guard Stream: Real-Time Token-Level Safety Moderation}, author={Hanzo AI and Zoo Labs Foundation}, year={2025}, publisher={HuggingFace}, howpublished={\url{https://huggingface.co/zenlm/zen-guard-stream}}}