GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
squ11z1
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
squ11z1
Model Tree
openai/gpt-oss-20b
Input Modalities
Output Modalities
Supported Functionality
GPT-OSS-Nano is a fine-tuned Mixture of Experts (MoE) language model optimized for step-by-step reasoning and problem solving. Built on the GPT-OSS architecture with sparse expert activation, it achieves strong reasoning performance while using only ~3B active parameters per forward pass.
| Feature | Description |
|---|---|
| 🧠 Sparse MoE | 12 experts, 4 active per token — efficient compute |
| 📝 Chain-of-Thought | Fine-tuned on reasoning datasets with step-by-step solutions |
| ⚡ 128K Context | Long context with YaRN rope scaling |
| 🔮 Q-GPT Ready | Compatible with quantum confidence estimation |
| 📦 GGUF Available | Run locally with llama.cpp or Ollama |
text
┌─────────────────────────────────────────────────────────┐│ GPT-OSS-Nano │├─────────────────────────────────────────────────────────┤│ Total Parameters │ 9.0 Billion ││ Active Parameters │ ~3 Billion (per forward pass) ││ Hidden Dimension │ 2880 ││ Attention Heads │ 64 (8 KV heads, GQA) ││ Layers │ 24 ││ Experts │ 12 total, 4 active ││ Context Length │ 131,072 tokens ││ Vocabulary Size │ 201,088 ││ Precision │ BFloat16 │└─────────────────────────────────────────────────────────┘python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model = AutoModelForCausalLM.from_pretrained( "squ11z1/gpt-oss-nano", torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,)tokenizer = AutoTokenizer.from_pretrained( "squ11z1/gpt-oss-nano", trust_remote_code=True,) prompt = """Solve this step by step:A store offers 20% off on all items. If a jacket costs $85, what is the final price after discount?""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate( **inputs, max_new_tokens=256, temperature=0.7, do_sample=True,)print(tokenizer.decode(outputs[0], skip_special_tokens=True))python
from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( "squ11z1/gpt-oss-nano", dtype=None, load_in_4bit=True, # 4-bit quantization for efficiency) # For inferenceFastLanguageModel.for_inference(model)bash
# Download the quantized modelwget https://huggingface.co/squ11z1/gpt-oss-nano/resolve/main/gpt-oss-9b-q4_k_m.gguf # Run inference./llama-cli -m gpt-oss-9b-q4_k_m.gguf \ -p "Solve step by step: What is 15% of 240?" \ -n 256 --temp 0.7bash
# Create Modelfileecho 'FROM ./gpt-oss-9b-q4_k_m.gguf' > Modelfileollama create gpt-oss-nano -f Modelfile # Runollama run gpt-oss-nano "Explain quantum computing simply"| Parameter | Value |
|---|---|
| Base Model | openai/gpt-oss-20b |
| Method | QLoRA (4-bit quantized LoRA) |
| LoRA Rank | 32 |
| LoRA Alpha | 32 |
| Learning Rate | 2e-4 |
| Batch Size | 2 (gradient accumulation: 8) |
| Epochs | 2 |
| Framework | Unsloth + TRL |
| Hardware | NVIDIA H200 |
Dataset: Superior-Reasoning — chain-of-thought examples with step-by-step problem solving.
GPT-OSS-Nano is compatible with Q-GPT — a quantum neural network that estimates response confidence.
python
from q_gpt import load_qgpt model, tokenizer = load_qgpt("squ11z1/gpt-oss-nano")outputs = model.generate_with_confidence(inputs, max_new_tokens=256) print(f"Response confidence: {outputs['confidence_label']}")# Output: "high", "moderate", "low", etc. if outputs['should_refuse']: print("⚠️ Model is uncertain — consider refusing to answer")Learn more: squ11z1/Q-GPT
This model is released under the Apache 2.0 License.
bibtex
@misc{gptossnano2026, title={GPT-OSS-Nano: Compact MoE Reasoning Model}, author={squ11z1}, year={2026}, publisher={Hugging Face}, url={https://huggingface.co/squ11z1/gpt-oss-nano}}Pro Mundi Vita
┌─────────────────────────────────────────────────────────┐│ GPT-OSS-Nano │├─────────────────────────────────────────────────────────┤│ Total Parameters │ 9.0 Billion ││ Active Parameters │ ~3 Billion (per forward pass) ││ Hidden Dimension │ 2880 ││ Attention Heads │ 64 (8 KV heads, GQA) ││ Layers │ 24 ││ Experts │ 12 total, 4 active ││ Context Length │ 131,072 tokens ││ Vocabulary Size │ 201,088 ││ Precision │ BFloat16 │└─────────────────────────────────────────────────────────┘from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model = AutoModelForCausalLM.from_pretrained( "squ11z1/gpt-oss-nano", torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,)tokenizer = AutoTokenizer.from_pretrained( "squ11z1/gpt-oss-nano", trust_remote_code=True,) prompt = """Solve this step by step:A store offers 20% off on all items. If a jacket costs $85, what is the final price after discount?""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate( **inputs, max_new_tokens=256, temperature=0.7, do_sample=True,)print(tokenizer.decode(outputs[0], skip_special_tokens=True))from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( "squ11z1/gpt-oss-nano", dtype=None, load_in_4bit=True, # 4-bit quantization for efficiency) # For inferenceFastLanguageModel.for_inference(model)# Download the quantized modelwget https://huggingface.co/squ11z1/gpt-oss-nano/resolve/main/gpt-oss-9b-q4_k_m.gguf # Run inference./llama-cli -m gpt-oss-9b-q4_k_m.gguf \ -p "Solve step by step: What is 15% of 240?" \ -n 256 --temp 0.7# Create Modelfileecho 'FROM ./gpt-oss-9b-q4_k_m.gguf' > Modelfileollama create gpt-oss-nano -f Modelfile # Runollama run gpt-oss-nano "Explain quantum computing simply"from q_gpt import load_qgpt model, tokenizer = load_qgpt("squ11z1/gpt-oss-nano")outputs = model.generate_with_confidence(inputs, max_new_tokens=256) print(f"Response confidence: {outputs['confidence_label']}")# Output: "high", "moderate", "low", etc. if outputs['should_refuse']: print("⚠️ Model is uncertain — consider refusing to answer")@misc{gptossnano2026, title={GPT-OSS-Nano: Compact MoE Reasoning Model}, author={squ11z1}, year={2026}, publisher={Hugging Face}, url={https://huggingface.co/squ11z1/gpt-oss-nano}}