GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
syaffers
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
syaffers
Model Tree
TinyModels/Atom-350M
Input Modalities
Output Modalities
Supported Functionality
Atom-350M-NVFP4 is the quantized version of TinyModels/Atom-350M, a compact open-source chat assistant fine-tuned from SmolLM2-360M-Instruct. Quantization was performed with llm-compressor using one-shot post-training quantization (PTQ). For more information on the base model, please check here.
This model is ready for commercial/non-commercial use.
This model is a quantized derivative of TinyModels/Atom-350M. See the Atom-350M Model Card for details on the base model's development and intended use.
Global
Developers looking for an extremely compact, pre-quantized model for on-device inference, edge deployment, chatbots, and other resource-constrained AI applications.
Hugging Face 06/03/2026 via https://huggingface.co/syaffers/Atom-350M-NVFP4
Architecture Type: Transformers Network Architecture: Decoder-only causal language model Number of Model Parameters: ~360M
Input Type(s): Text Input Format(s): String Input Parameters: 1D (One Dimensional) Other Properties Related to Input: Supports multi-turn conversations formatted via the model's chat template. Maximum context length of 8192 tokens.
Output Type(s): Text Output Format: String Output Parameters: 1D (One Dimensional)
Supported Runtime Engine(s):
Preferred Operating System(s):
The model is quantized to NVFP4 using llmcompressor.
Link: HuggingFaceH4/ultrachat_200k
Split: train_sft
Samples used: 1024
Data Collection Method: Automated
Labeling Method: Automated
Properties: UltraChat 200k is a large-scale, high-quality dataset of multi-turn conversational exchanges. Samples were formatted using the model's chat template and tokenized with truncation at 8192 tokens.
Acceleration Engine: Hugging Face Transformers, vLLM Test Hardware: NVIDIA GPU (CUDA)
This model was obtained by quantizing the weights and activations of Atom-350M to the NVFP4 data type using one-shot PTQ via llm-compressor. All Linear layers within transformer blocks are quantized; the lm_head is excluded and kept at full precision to preserve output distribution.
python
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "syaffers/Atom-350M-NVFP4", torch_dtype="auto", device_map="auto",)tokenizer = AutoTokenizer.from_pretrained("syaffers/Atom-350M-NVFP4") messages = [{"role": "user", "content": "Explain how a bicycle stays upright in simple terms."}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response)Tested on vLLM v0.21.0 or later.
python
from vllm import LLM, SamplingParams llm = LLM(model="syaffers/Atom-350M-NVFP4", quantization="modelopt") sampling_params = SamplingParams(temperature=0.7, max_tokens=150) messages = [{"role": "user", "content": "Explain how a bicycle stays upright in simple terms."}]outputs = llm.chat(messages, sampling_params=sampling_params)print(outputs[0].outputs[0].text)Start the server:
sh
vllm serve syaffers/Atom-350M-NVFP4 --quantization modelopt --port 8000Then query it with the OpenAI client or any HTTP client:
python
from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="placeholder") response = client.chat.completions.create( model="syaffers/Atom-350M-NVFP4", messages=[{"role": "user", "content": "Explain how a bicycle stays upright in simple terms."}], max_tokens=150, temperature=0.7,)print(response.choices[0].message.content)The base model was fine-tuned on data that may contain biases present in the source corpora. The NVFP4 quantization introduces a small degree of approximation error that may slightly affect output quality on edge cases. This model is not intended for safety-critical applications without additional guardrails.
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "syaffers/Atom-350M-NVFP4", torch_dtype="auto", device_map="auto",)tokenizer = AutoTokenizer.from_pretrained("syaffers/Atom-350M-NVFP4") messages = [{"role": "user", "content": "Explain how a bicycle stays upright in simple terms."}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response)from vllm import LLM, SamplingParams llm = LLM(model="syaffers/Atom-350M-NVFP4", quantization="modelopt") sampling_params = SamplingParams(temperature=0.7, max_tokens=150) messages = [{"role": "user", "content": "Explain how a bicycle stays upright in simple terms."}]outputs = llm.chat(messages, sampling_params=sampling_params)print(outputs[0].outputs[0].text)from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="placeholder") response = client.chat.completions.create( model="syaffers/Atom-350M-NVFP4", messages=[{"role": "user", "content": "Explain how a bicycle stays upright in simple terms."}], max_tokens=150, temperature=0.7,)print(response.choices[0].message.content)vllm serve syaffers/Atom-350M-NVFP4 --quantization modelopt --port 8000