VertexAGI

quartz-micro-preview-v3

README

License: apache-2.0

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Table with columns: Format, File / folder, Precision, Size, Works with
Format	File / folder	Precision	Size	Works with
Transformers (safetensors)	`model.safetensors` (repo root)	fp32 (32-bit)	400 MB	Hugging Face Transformers / PyTorch
MLX	`mlx/fp16/`	fp16 (16-bit)	200 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q4/`	4-bit (group size 64)	63 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q8/`	8-bit (group size 64)	113 MB	mlx-lm / MLX on Apple Silicon
GGUF	`gguf/quartz-micro-preview-v3-f16.gguf`	F16 (16-bit float)	201 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v3-q4_k_m.gguf`	Q4_K_M (4-bit)	67 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v3-q8_0.gguf`	Q8_0 (8-bit)	108 MB	llama.cpp, Ollama, LM Studio

Formats (one repo, three formats)

Table with columns: Where, Format
Where	Format
repo root	stock Hugging Face `LlamaForCausalLM` (fp32 safetensors), loads with plain `transformers` and plain `mlx_lm`
`mlx/fp16`, `mlx/q8`, `mlx/q4`	stock MLX (fp16, 8-bit, 4-bit)
`gguf/`	stock llama.cpp GGUF: `f16`, `q8_0`, `q4_k_m`

Every format was checked against the original training code on held-out text (perplexity; lower is better):

Table with columns: Format, Perplexity, Difference from original
Format	Perplexity	Difference from original
transformers fp32 (this repo root)	13.878	0.000%
MLX fp16	13.875	0.020%
MLX 8-bit	13.878	0.001%
MLX 4-bit	14.739	6.205%
GGUF f16	13.878	0.001%
GGUF Q8_0	13.877	0.009%
GGUF Q4_K_M	13.955	0.557%

Usage

python

# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v3")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v3")messages = [{"role": "user", "content": "Why do cats purr?"}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=400, do_sample=True, temperature=0.6, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

python

# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v3")print(generate(model, tok, prompt=tok.apply_chat_template([{"role": "user", "content": "Why do cats purr?"}], add_generation_prompt=True), max_tokens=400))

bash

# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v3 gguf/quartz-micro-preview-v3-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v3-q8_0.gguf -cnv

Benchmark: V3 vs V2 (instruct models, same prompts)

Both models answered the same 156 prompts with identical decoding (greedy, repetition penalty 1.1, same tokenizer and chat template), run on one GPU. Judge for the open-ended parts: nvidia/nemotron-3-ultra-550b-a55b, temperature 0; pairwise comparisons were judged in both orders and only count as a win when both orders agree. No judge call failed (all counted, none dropped).

Table with columns: Test, V2, V3
Test	V2	V3
Identity, 100 held-out prompts (judge checks the reply against the model's fact sheet)	3/100 (3%)	71/100 (71%)
Identity, 24 hand-written prompts	5/24 (21%)	23/24 (96%)
Facts, V2's 15 original keyword prompts	5/15	11/15
Facts, 11 new prompts (grounded QA, honest limits, simple facts)	4/11	9/11
Facts, all 26	9/26	20/26
Head-to-head on 33 general prompts (judge, both orders agree)	4 wins	25 wins (4 ties)
Answers that end on their own (`<eos>`)	23/33	31/33
Repetition (distinct-4-gram, higher = less repetitive)	0.801	0.97
Average answer length (words)	126.8	51.5
Loss on assistant tokens, general validation set (lower is better)	2.383	1.719
Loss on held-out identity answers (lower is better)	4.298	1.304

V3 vs V2

How to read this

Identity is where V3 was designed to improve (V3 adds the quartz-identity-data set; V2 had none). V2 usually names a different maker (it called itself "Open Assistant" in earlier tests). The held-out set is 100 prompts from a split that was never in V3's training data; V3 still fails 29 of them, mostly in multi-turn conversations (4/13 passed).
Facts is a keyword check (the answer must contain the expected word), so it rewards plain correct answers and is easy to fool; it is a smoke test, not a knowledge benchmark. The 11 "new" prompts target what V3's extra data teaches (answer from a passage, say "the passage does not say", admit it cannot browse), so V2 was never trained for them: that row shows the added ability, not a like-for-like gain.
Length: V2 rambles (127 words on average, 10 of 33 answers never ended within the 300-token budget); V3 is shorter (52 words) and mostly stops on its own.
Still weak: V3 is a 100M-parameter model. It still invents facts (for example it described a made-up city as fictional and from a famous novel, and wrote a 16-line poem when asked for two). Treat answers as unreliable.
The two runs are a single greedy sample per prompt; with 26 keyword prompts and 33 judged comparisons, differences of one or two items are noise. Judge scores moved by 1 item between two scoring passes of the same V3 answers (70 vs 71 of 100).
V2 and V3 differ in more than the instruction data: V3 is pretrained on a much larger mix (about 6.5 B tokens vs 100 M for V2), so this compares the two releases, not the effect of the SFT data alone.

Every prompt and both models' replies are in eval_outputs_v3_vs_v2.md; the raw numbers are in eval_scores_v3_vs_v2.json.

Instruction tuning

Base: VertexAGI/quartz-micro-preview-v3-base (pretrained on ~6.5B tokens, see that card).
Data: 44,389 English conversations (12.3M tokens; 7.6M are assistant tokens that carry the loss): the V2 chat set (24,170 conversations, OpenAssistant OASST2 + smol-smoltalk slices), Dolly-15k (11,881), SQuAD 2.0 grounded questions with the unanswerable ones answered "the passage does not say" (3,908), CoCoNot "honest limits" examples (no browsing, no real-time information, false premises; 1,730) and 2,700 identity conversations that teach the model who it is. 300 further identity conversations were held out and never trained on.
Method: full-parameter fine-tune, loss on assistant tokens only, 2 epochs (886 steps), lr 1e-4 cosine, fp32 on the same GPU (about 1.5 hours). Final validation loss on the held-out chat set: 1.719 (the pretrained model: 2.085).
Chat format: <|user|>\n{message}\n<|assistant|>\n{reply}<eos> (built into the tokenizer's chat template; no system role, so put instructions in the user message).
Suggested sampling: temperature 0.6, top-p 0.9, repetition penalty 1.1.

Identity

V3 was tuned to say that it is a small language model from Vertex AGI in the Quartz series, a preview that is not reliable, with no browsing, no memory between chats and no image or audio input, and that it is not ChatGPT, Claude, Gemini or another assistant. It is right about this in most single questions (71 of 100 held-out prompts, 23 of 24 hand-written ones), and weaker in longer conversations. It can still be argued out of it.

Limitations

States wrong facts and invents details (for example it described a made-up city as fictional and from a famous novel, and wrote a 16-line poem when asked for two).
Weak at maths, logic and code; follows length and format limits unreliably.
1,024-token context; English only; no internet, tools, images or audio.
No safety tuning or refusal training (the CoCoNot examples teach it to admit what it cannot do, not to refuse harmful requests). Do not deploy it unsupervised.

Data and licences (read before commercial use)

The weights are released under Apache-2.0, but they were trained on third-party data with its own terms, and some of those terms are NonCommercial or ShareAlike (for example the Tulu-3 mixture includes no_robots, CC-BY-NC; Wikipedia, Stack Exchange, Dolly and SQuAD are CC-BY-SA; CoCoNot is ODC-By). Whether training-data terms extend to model weights is legally unsettled. This is a research / hobby model: if you need legal certainty for a commercial product, ask a lawyer or retrain on data you have cleared. The pretraining and identity datasets are not public.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

VertexAGI

Model Tree

Base

VertexAGI/quartz-micro-preview-v3-base

Quantized

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Table with columns: Format, File / folder, Precision, Size, Works with
Format	File / folder	Precision	Size	Works with
Transformers (safetensors)	`model.safetensors` (repo root)	fp32 (32-bit)	400 MB	Hugging Face Transformers / PyTorch
MLX	`mlx/fp16/`	fp16 (16-bit)	200 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q4/`	4-bit (group size 64)	63 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q8/`	8-bit (group size 64)	113 MB	mlx-lm / MLX on Apple Silicon
GGUF	`gguf/quartz-micro-preview-v3-f16.gguf`	F16 (16-bit float)	201 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v3-q4_k_m.gguf`	Q4_K_M (4-bit)	67 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v3-q8_0.gguf`	Q8_0 (8-bit)	108 MB	llama.cpp, Ollama, LM Studio

Formats (one repo, three formats)

Table with columns: Where, Format
Where	Format
repo root	stock Hugging Face `LlamaForCausalLM` (fp32 safetensors), loads with plain `transformers` and plain `mlx_lm`
`mlx/fp16`, `mlx/q8`, `mlx/q4`	stock MLX (fp16, 8-bit, 4-bit)
`gguf/`	stock llama.cpp GGUF: `f16`, `q8_0`, `q4_k_m`

Every format was checked against the original training code on held-out text (perplexity; lower is better):

Table with columns: Format, Perplexity, Difference from original
Format	Perplexity	Difference from original
transformers fp32 (this repo root)	13.878	0.000%
MLX fp16	13.875	0.020%
MLX 8-bit	13.878	0.001%
MLX 4-bit	14.739	6.205%
GGUF f16	13.878	0.001%
GGUF Q8_0	13.877	0.009%
GGUF Q4_K_M	13.955	0.557%

Usage

python

# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v3")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v3")messages = [{"role": "user", "content": "Why do cats purr?"}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=400, do_sample=True, temperature=0.6, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v3")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v3")messages = [{"role": "user", "content": "Why do cats purr?"}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=400, do_sample=True, temperature=0.6, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

python

# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v3")print(generate(model, tok, prompt=tok.apply_chat_template([{"role": "user", "content": "Why do cats purr?"}], add_generation_prompt=True), max_tokens=400))

# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v3")print(generate(model, tok, prompt=tok.apply_chat_template([{"role": "user", "content": "Why do cats purr?"}], add_generation_prompt=True), max_tokens=400))

bash

# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v3 gguf/quartz-micro-preview-v3-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v3-q8_0.gguf -cnv

# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v3 gguf/quartz-micro-preview-v3-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v3-q8_0.gguf -cnv

Benchmark: V3 vs V2 (instruct models, same prompts)

Table with columns: Test, V2, V3
Test	V2	V3
Identity, 100 held-out prompts (judge checks the reply against the model's fact sheet)	3/100 (3%)	71/100 (71%)
Identity, 24 hand-written prompts	5/24 (21%)	23/24 (96%)
Facts, V2's 15 original keyword prompts	5/15	11/15
Facts, 11 new prompts (grounded QA, honest limits, simple facts)	4/11	9/11
Facts, all 26	9/26	20/26
Head-to-head on 33 general prompts (judge, both orders agree)	4 wins	25 wins (4 ties)
Answers that end on their own (`<eos>`)	23/33	31/33
Repetition (distinct-4-gram, higher = less repetitive)	0.801	0.97
Average answer length (words)	126.8	51.5
Loss on assistant tokens, general validation set (lower is better)	2.383	1.719
Loss on held-out identity answers (lower is better)	4.298	1.304

V3 vs V2

How to read this

Identity is where V3 was designed to improve (V3 adds the quartz-identity-data set; V2 had none). V2 usually names a different maker (it called itself "Open Assistant" in earlier tests). The held-out set is 100 prompts from a split that was never in V3's training data; V3 still fails 29 of them, mostly in multi-turn conversations (4/13 passed).
Facts is a keyword check (the answer must contain the expected word), so it rewards plain correct answers and is easy to fool; it is a smoke test, not a knowledge benchmark. The 11 "new" prompts target what V3's extra data teaches (answer from a passage, say "the passage does not say", admit it cannot browse), so V2 was never trained for them: that row shows the added ability, not a like-for-like gain.
Length: V2 rambles (127 words on average, 10 of 33 answers never ended within the 300-token budget); V3 is shorter (52 words) and mostly stops on its own.
Still weak: V3 is a 100M-parameter model. It still invents facts (for example it described a made-up city as fictional and from a famous novel, and wrote a 16-line poem when asked for two). Treat answers as unreliable.
The two runs are a single greedy sample per prompt; with 26 keyword prompts and 33 judged comparisons, differences of one or two items are noise. Judge scores moved by 1 item between two scoring passes of the same V3 answers (70 vs 71 of 100).
V2 and V3 differ in more than the instruction data: V3 is pretrained on a much larger mix (about 6.5 B tokens vs 100 M for V2), so this compares the two releases, not the effect of the SFT data alone.

Every prompt and both models' replies are in eval_outputs_v3_vs_v2.md; the raw numbers are in eval_scores_v3_vs_v2.json.

Instruction tuning

Base: VertexAGI/quartz-micro-preview-v3-base (pretrained on ~6.5B tokens, see that card).
Data: 44,389 English conversations (12.3M tokens; 7.6M are assistant tokens that carry the loss): the V2 chat set (24,170 conversations, OpenAssistant OASST2 + smol-smoltalk slices), Dolly-15k (11,881), SQuAD 2.0 grounded questions with the unanswerable ones answered "the passage does not say" (3,908), CoCoNot "honest limits" examples (no browsing, no real-time information, false premises; 1,730) and 2,700 identity conversations that teach the model who it is. 300 further identity conversations were held out and never trained on.
Method: full-parameter fine-tune, loss on assistant tokens only, 2 epochs (886 steps), lr 1e-4 cosine, fp32 on the same GPU (about 1.5 hours). Final validation loss on the held-out chat set: 1.719 (the pretrained model: 2.085).
Chat format: <|user|>\n{message}\n<|assistant|>\n{reply}<eos> (built into the tokenizer's chat template; no system role, so put instructions in the user message).
Suggested sampling: temperature 0.6, top-p 0.9, repetition penalty 1.1.

Identity

Limitations

States wrong facts and invents details (for example it described a made-up city as fictional and from a famous novel, and wrote a 16-line poem when asked for two).
Weak at maths, logic and code; follows length and format limits unreliably.
1,024-token context; English only; no internet, tools, images or audio.
No safety tuning or refusal training (the CoCoNot examples teach it to admit what it cannot do, not to refuse harmful requests). Do not deploy it unsupervised.