GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
VertexAGI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
VertexAGI
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Everything below lives in this one repository: pick the format that matches your runtime.
| Format | File / folder | Precision | Size | Works with |
|---|---|---|---|---|
| Transformers (safetensors) | model.safetensors (repo root) | fp32 (32-bit) | 400 MB | Hugging Face Transformers / PyTorch |
| MLX | mlx/fp16/ | fp16 (16-bit) | 200 MB | mlx-lm / MLX on Apple Silicon |
| MLX | mlx/q4/ | 4-bit (group size 64) | 63 MB | mlx-lm / MLX on Apple Silicon |
| MLX | mlx/q8/ | 8-bit (group size 64) | 113 MB | mlx-lm / MLX on Apple Silicon |
| GGUF | gguf/quartz-micro-preview-v2-base-f16.gguf | F16 (16-bit float) | 201 MB | llama.cpp, Ollama, LM Studio |
| GGUF | gguf/quartz-micro-preview-v2-base-q4_k_m.gguf | Q4_K_M (4-bit) | 67 MB | llama.cpp, Ollama, LM Studio |
| GGUF | gguf/quartz-micro-preview-v2-base-q8_0.gguf | Q8_0 (8-bit) | 108 MB | llama.cpp, Ollama, LM Studio |
| Where | Format |
|---|---|
| repo root | stock Hugging Face LlamaForCausalLM (fp32 safetensors), loads with plain transformers and plain mlx_lm |
mlx/fp16, mlx/q8, mlx/q4 | stock MLX (fp16, 8-bit, 4-bit) |
gguf/ | stock llama.cpp GGUF: f16, q8_0, q4_k_m |
Every format was checked against the original training code on held-out text (perplexity; lower is better):
| Format | Perplexity | Difference from original |
|---|---|---|
| transformers fp32 (this repo root) | 21.577 | 0.000% |
| MLX fp16 | 21.577 | 0.001% |
| MLX 8-bit | 21.578 | 0.004% |
| MLX 4-bit | 22.151 | 2.659% |
| GGUF f16 | 21.577 | 0.001% |
| GGUF Q8_0 | 21.598 |
python
# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v2-base")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v2-base")ids = tok("The history of the Roman Empire", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0], skip_special_tokens=True))python
# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v2-base")print(generate(model, tok, prompt="The history of the Roman Empire", max_tokens=60))bash
# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v2-base gguf/quartz-micro-preview-v2-base-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v2-base-q8_0.gguf -p "The history of the Roman Empire" -n 60 -no-cnveval_outputs.md, including the wrong and repetitive ones: it continues topics fluently but states wrong facts and loops (for example on "The three states of matter are").No instruction following, no safety tuning, English only, 1,024-token context, no benchmark decontamination of the pretraining data, and it inherits errors and biases from web and synthetic text.
Apache-2.0 for the weights and code. Training data carries its own terms (see the dataset cards; Wikipedia and Stack Exchange are share-alike).
| 0.095% |
| GGUF Q4_K_M | 21.641 | 0.296% |