GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
teolm30
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
teolm30
Model Tree
Qwen/Qwen2.5-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
The repository includes a Q8_0 quantized GGUF file for ultra-fast CPU inference with llama.cpp, Ollama, LM Studio, or any GGUF-compatible runner:
| File | Size | Format | Quality |
|---|---|---|---|
Ult1.0-Q8_0.gguf | 3.29 GB | Q8_0 (8-bit) | Near-lossless |
bash
./llama-cli -m Ult1.0-Q8_0.gguf -p "Write a poem about AI" -n 256
bash
ollama create ult1.0 -f Modelfile# Modelfile content: FROM ./Ult1.0-Q8_0.ggufollama run ult1.0
python
from llama_cpp import Llamallm = Llama("Ult1.0-Q8_0.gguf", n_ctx=32768)output = llm("Write a poem about AI", max_tokens=256)print(output["choices"][0]["text"])
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel = AutoModelForCausalLM.from_pretrained("teolm30/Ult1.0", device_map="auto")tokenizer = AutoTokenizer.from_pretrained("teolm30/Ult1.0")messages = [{"role": "user", "content": "Explain quantum computing simply"}]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=256)print(tokenizer.decode(outputs[0], skip_special_tokens=True))
| Metric | Full Fine-Tune | Ult1.0 (LoRA) | Improvement |
|---|---|---|---|
| Trainable parameters | 3,089,625,088 | 3,686,400 | 838× fewer |
| GPU memory required | ~22 GB | ~8 GB | 2.8× less |
| Storage size | ~6 GB | ~15 MB | 400× smaller |
| Training time (3 epochs) | ~3 days | ~4 hours |
Fine-tune on any GPU with ≥8 GB VRAM:
bash
pip install transformers datasets peft acceleratepython train.py
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-3B-Instruct |
| Total Parameters | 3,089,625,088 |
| LoRA Parameters | 3,686,400 (0.12%) |
| LoRA Rank | 8 |
| Context Length | 32,768 tokens |
| Architecture | Transformer with RoPE, SwiGLU, Grouped Query Attention |
| 18× faster |