GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
fluently
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
fluently
Model Tree
fluently/FluentlyQwen3-4B
Input Modalities
Output Modalities
Supported Functionality
*The recipe is approximate, there are some inaccuracies.
| Task | Result |
|---|---|
| Basic Communication | Improved |
| Translation | Improved |
| Mathematics | Improved |
| Physics | Improved |
| Biology | Improved |
| Medicine | Improved |
| Coding | Improved |
| Agent Functions | Improved |
Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.
py
from transformers import AutoModelForCausalLM, AutoTokenizermodel_name = "fluently/FluentlyQwen3-Coder-4B"# load the tokenizer and the modeltokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto")# prepare the model inputprompt = "Give me a short introduction to large language model."messages = [ {"role": "user", "content": prompt}]text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.)model_inputs = tokenizer([text], return_tensors="pt").to(model.device)# conduct text completiongenerated_ids = model.generate( **model_inputs, max_new_tokens=32768)output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() # parsing thinking contenttry: # rindex finding 151668 (</think>) index = len(output_ids) - output_ids[::-1].index(151668)except ValueError: index = 0thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")print("thinking content:", thinking_content)print("content:", content)For local use, applications such as Ollama, LMStudio, MLX-LM, llama.cpp, and KTransformers have also supported Qwen3.
[!TIP] The
enable_thinkingswitch is also available in APIs created by SGLang and vLLM.
enable_thinking=TrueBy default, Qwen3 has thinking capabilities enabled, similar to QwQ-32B. This means the model will use its reasoning abilities to enhance the quality of generated responses. For example, when explicitly setting enable_thinking=True or leaving it as the default value in tokenizer.apply_chat_template, the model will engage its thinking mode.
python
text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True # True is the default value for enable_thinking)In this mode, the model will generate think content wrapped in a <think>...</think> block, followed by the final response.
[!NOTE] For thinking mode, use
Temperature=0.6,TopP=0.95,TopK=20andMinP=0(the default setting ingeneration_config.json). DO NOT use greedy decoding, as it can lead to performance degradation and endless repetitions.
enable_thinking=FalseWe provide a hard switch to strictly disable the model's thinking behavior, aligning its functionality with the previous Qwen2.5-Instruct models. This mode is particularly useful in scenarios where disabling thinking is essential for enhancing efficiency.
python
text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False # Setting enable_thinking=False disables thinking mode)In this mode, the model will not generate any think content and will not include a <think>...</think> block.
[!NOTE] For non-thinking mode, we suggest using
Temperature=0.7,TopP=0.8,TopK=20, andMinP=0.
🤗 We are grateful for open source resources, technologies and assistance from:
from transformers import AutoModelForCausalLM, AutoTokenizermodel_name = "fluently/FluentlyQwen3-Coder-4B"# load the tokenizer and the modeltokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto")# prepare the model inputprompt = "Give me a short introduction to large language model."messages = [ {"role": "user", "content": prompt}]text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.)model_inputs = tokenizer([text], return_tensors="pt").to(model.device)# conduct text completiongenerated_ids = model.generate( **model_inputs, max_new_tokens=32768)output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() # parsing thinking contenttry: # rindex finding 151668 (</think>) index = len(output_ids) - output_ids[::-1].index(151668)except ValueError: index = 0thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")print("thinking content:", thinking_content)print("content:", content)text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True # True is the default value for enable_thinking)text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False # Setting enable_thinking=False disables thinking mode)