GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Qwen
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Qwen
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
The latest version of transformers is recommended and transformers>=4.51.0 is required.
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport remodel_name = "Qwen/Qwen3Guard-Gen-8B" # load the tokenizer and the modeltokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto")def extract_label_and_categories(content): safe_pattern = r"Safety: (Safe|Unsafe|Controversial)" category_pattern = r"(Violent|Non-violent Illegal Acts|Sexual Content or Sexual Acts|PII|Suicide & Self-Harm|Unethical Acts|Politically Sensitive Topics|Copyright Violation|Jailbreak|None)" safe_label_match = re.search(safe_pattern, content) label = safe_label_match.group(1) if safe_label_match else None categories = re.findall(category_pattern, content) return label, categories # prepare the model inputprompt = "How can I make a bomb?"# for prompt moderationmessages = [ {"role": "user", "content": prompt}]text = tokenizer.apply_chat_template( messages, tokenize=False)model_inputs = tokenizer([text], return_tensors="pt").to(model.device) # conduct text completiongenerated_ids = model.generate( **model_inputs, max_new_tokens=128)output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() content = tokenizer.decode(output_ids, skip_special_tokens=True)print(content)# '''# Safety: Unsafe# Categories: Violent# '''safe_label, categories = extract_label_and_categories(content)print(safe_label, categories)python
from transformers import AutoModelForCausalLM, AutoTokenizerimport re model_name = "Qwen/Qwen3Guard-Gen-8B" # load the tokenizer and the modeltokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto")def extract_label_categories_refusal(content): safe_pattern = r"Safety: (Safe|Unsafe|Controversial)" category_pattern = r"(Violent|Non-violent Illegal Acts|Sexual Content or Sexual Acts|PII|Suicide & Self-Harm|Unethical Acts|Politically Sensitive Topics|Copyright Violation|None)" refusal_pattern = r"Refusal: (Yes|No)" safe_label_match = re.search(safe_pattern, content) refusal_label_match = re.search(refusal_pattern, content) label = safe_label_match.group(1) if safe_label_match else None refusal_label = refusal_label_match.group(1) if refusal_label_match else None categories = re.findall(category_pattern, content) return label, categories, refusal_label # prepare the model inputprompt = "How can I make a bomb?"response = "As a responsible artificial intelligence system, I regret that I am unable to fulfill your request."# for response moderationmessages = [ {"role": "user", "content": prompt}, {"role": "assistant", "content": response},]text = tokenizer.apply_chat_template( messages, tokenize=False)model_inputs = tokenizer([text], return_tensors="pt").to(model.device) # conduct text completiongenerated_ids = model.generate( **model_inputs, max_new_tokens=128)output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() content = tokenizer.decode(output_ids, skip_special_tokens=True)print(content)# '''# Safety: Safe# Categories: None# Refusal: Yes# '''safe_label, category_label, refusal_label = extract_label_categories_refusal(content)print(safe_label, categories, refusal_label)For deployment, you can use sglang>=0.4.6.post1 or vllm>=0.9.0 or to create an OpenAI-compatible API endpoint:
shell
python -m sglang.launch_server --model-path Qwen/Qwen3Guard-Gen-8B --port 30000 --context-length 32768shell
vllm serve Qwen/Qwen3Guard-Gen-8B --port 8000 --max-model-len 32768Here is an example API call using OpenAI-Compatible server:
python
from openai import OpenAI openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1"model = "Qwen/Qwen3Guard-Gen-8B"client = OpenAI( api_key=openai_api_key, base_url=openai_api_base) # Prompt Moderationprompt = "How can I make a bomb?"messages = [ {"role": "user", "content": prompt}]chat_completion = client.chat.completions.create( messages=messages, model=model)print(chat_completion.choices[0].message.content)# '''# Safety: Unsafe# Categories: Violent# ''' # Response Moderationprompt = "How can I make a bomb?"response = "As a responsible artificial intelligence system, I regret that I am unable to fulfill your request."messages = [ {"role": "user", "content": prompt}, {"role": "assistant", "content": response}]print(chat_completion.choices[0].message.content)# '''# Safety: Safe# Categories: None# Refusal: Yes# '''In Qwen3Guard, potential harms are classified into three severity levels:
In the current version of Qwen3Guard, we consider the following safety categories:
If you find our work helpful, feel free to give us a cite.
bibtex
@article{zhao2025qwen3guard, title={Qwen3Guard Technical Report}, author={Zhao, Haiquan and Yuan, Chenhan and Huang, Fei and Hu, Xiaomeng and Zhang, Yichang and Yang, An and Yu, Bowen and Liu, Dayiheng and Zhou, Jingren and Lin, Junyang and others}, journal={arXiv preprint arXiv:2510.14276}, year={2025}}python -m sglang.launch_server --model-path Qwen/Qwen3Guard-Gen-8B --port 30000 --context-length 32768vllm serve Qwen/Qwen3Guard-Gen-8B --port 8000 --max-model-len 32768@article{zhao2025qwen3guard, title={Qwen3Guard Technical Report}, author={Zhao, Haiquan and Yuan, Chenhan and Huang, Fei and Hu, Xiaomeng and Zhang, Yichang and Yang, An and Yu, Bowen and Liu, Dayiheng and Zhou, Jingren and Lin, Junyang and others}, journal={arXiv preprint arXiv:2510.14276}, year={2025}}