GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
fluently
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
fluently
Model Tree
fluently/FluentlyQwen3-4B
Input Modalities
Output Modalities
Supported Functionality
*The recipe is approximate, there are some inaccuracies.
| Task | Result |
|---|---|
| Basic Communication | Improved |
| Translation | Improved |
| Mathematics | Improved |
| Physics | Improved |
| Biology | Improved |
| Medicine | Improved |
| Coding | Improved |
| Agent Functions | Improved |
Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.
py
from transformers import AutoModelForCausalLM, AutoTokenizermodel_name = "fluently/FluentlyQwen3-Coder-4B"# load the tokenizer and the modeltokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto")# prepare the model inputprompt = "Give me a short introduction to large language model."messages = [ {"role": "user", "content": prompt}]text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.)model_inputs = tokenizer([text], return_tensors="pt").to(model.device)# conduct text completiongenerated_ids = model.generate( **model_inputs, max_new_tokens=32768)output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() # parsing thinking contenttry: # rindex finding 151668 (</think>) index = len(output_ids) - output_ids[::-1].index(151668)except ValueError: index = 0thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")print("thinking content:", thinking_content)print("content:", content)For local use, applications such as Ollama, LMStudio, MLX-LM, llama.cpp, and KTransformers have also supported Qwen3.
[!TIP] The
enable_thinkingswitch is also available in APIs created by SGLang and vLLM.
enable_thinking=TrueBy default, Qwen3 has thinking capabilities enabled, similar to QwQ-32B. This means the model will use its reasoning abilities to enhance the quality of generated responses. For example, when explicitly setting enable_thinking=True or leaving it as the default value in tokenizer.apply_chat_template, the model will engage its thinking mode.
python
text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True # True is the default value for enable_thinking)In this mode, the model will generate think content wrapped in a <think>...</think> block, followed by the final response.
[!NOTE] For thinking mode, use
Temperature=0.6,TopP=0.95,TopK=20andMinP=0(the default setting ingeneration_config.json). DO NOT use greedy decoding, as it can lead to performance degradation and endless repetitions.
enable_thinking=FalseWe provide a hard switch to strictly disable the model's thinking behavior, aligning its functionality with the previous Qwen2.5-Instruct models. This mode is particularly useful in scenarios where disabling thinking is essential for enhancing efficiency.
python
text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False # Setting enable_thinking=False disables thinking mode)In this mode, the model will not generate any think content and will not include a <think>...</think> block.
[!NOTE] For non-thinking mode, we suggest using
Temperature=0.7,TopP=0.8,TopK=20, andMinP=0.
🤗 We are grateful for open source resources, technologies and assistance from:
text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True # True is the default value for enable_thinking)text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False # Setting enable_thinking=False disables thinking mode)from transformers import AutoModelForCausalLM, AutoTokenizermodel_name = "fluently/FluentlyQwen3-Coder-4B"# load the tokenizer and the modeltokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto")# prepare the model inputprompt = "Give me a short introduction to large language model."messages = [ {"role": "user", "content": prompt}]text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.)model_inputs = tokenizer([text], return_tensors="pt").to(model.device)# conduct text completiongenerated_ids = model.generate( **model_inputs, max_new_tokens=32768)output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() # parsing thinking contenttry: # rindex finding 151668 (</think>) index = len(output_ids) - output_ids[::-1].index(151668)except ValueError: index = 0thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")print("thinking content:", thinking_content)print("content:", content)