VertexAGI

quartz-micro-preview-v2

README

License: apache-2.0

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Table with columns: Format, File / folder, Precision, Size, Works with
Format	File / folder	Precision	Size	Works with
Transformers (safetensors)	`model.safetensors` (repo root)	fp32 (32-bit)	400 MB	Hugging Face Transformers / PyTorch
MLX	`mlx/fp16/`	fp16 (16-bit)	200 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q4/`	4-bit (group size 64)	63 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q8/`	8-bit (group size 64)	113 MB	mlx-lm / MLX on Apple Silicon
GGUF	`gguf/quartz-micro-preview-v2-f16.gguf`	F16 (16-bit float)	201 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v2-q4_k_m.gguf`	Q4_K_M (4-bit)	67 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v2-q8_0.gguf`	Q8_0 (8-bit)	108 MB	llama.cpp, Ollama, LM Studio

Formats (one repo, three formats)

Table with columns: Where, Format
Where	Format
repo root	stock Hugging Face `LlamaForCausalLM` (fp32 safetensors), loads with plain `transformers` and plain `mlx_lm`
`mlx/fp16`, `mlx/q8`, `mlx/q4`	stock MLX (fp16, 8-bit, 4-bit)
`gguf/`	stock llama.cpp GGUF: `f16`, `q8_0`, `q4_k_m`

Every format was checked against the original training code on held-out text (perplexity; lower is better):

Table with columns: Format, Perplexity, Difference from original
Format	Perplexity	Difference from original
transformers fp32 (this repo root)	27.844	0.000%
MLX fp16	27.842	0.007%
MLX 8-bit	27.857	0.045%
MLX 4-bit	28.457	2.202%
GGUF f16	27.844	0.000%
GGUF Q8_0	27.850

Usage

python

# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v2")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v2")messages = [{"role": "user", "content": "Why do cats purr?"}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=400, do_sample=True, temperature=0.6, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

python

# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v2")print(generate(model, tok, prompt=tok.apply_chat_template([{"role": "user", "content": "Why do cats purr?"}], add_generation_prompt=True), max_tokens=400))

bash

# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v2 gguf/quartz-micro-preview-v2-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v2-q8_0.gguf -cnv

Instruction tuning

Base: Quartz Micro Preview V2 Base (VertexAGI/quartz-micro-preview-v2-base).
Data: 24,170 English conversations (9,258,981 tokens): filtered OpenAssistant OASST2 (reviewed, non-synthetic, top-ranked replies, follow-up turns kept) plus short conversational / rewriting / summarizing / constraint-following slices of smol-smoltalk. Long reasoning, maths, code and URLs were removed.
Method: full-parameter fine-tune, loss on assistant tokens only, 2 epochs, lr 1e-4 cosine, fp32.
Chat format: <|user|>\n{message}\n<|assistant|>\n{reply}<eos> (built into the tokenizer's chat template; no system role - put instructions in the user message).
Suggested sampling: temperature 0.6, top-p 0.9, repetition penalty 1.1.

Evaluation (hand-written prompts, not copied from the training sets)

20/20 prompts answered, 17/20 replies ended cleanly with <eos>, mean distinct-4-gram ratio 0.79.
Simple keyword fact checks: 5/15 (a deliberately easy test; misses are normal for this size).
Assistant-token loss on held-out chat data: base 2.856 -> tuned 2.280.
All prompts and replies are in eval_outputs.md, including the wrong ones. Machine-readable results: eval_results.json (per-prompt replies, gate checks) and eval_formats.json (all shipped formats).

Same prompts, every shipped format (greedy, repetition penalty 1.1, 400 new tokens)

Table with columns: Format, Ended with, Keyword fact checks, Distinct-4-gram
Format	Ended with	Keyword fact checks	Distinct-4-gram
MLX fp16	13/21	6/15	0.57
MLX 8-bit	12/21	6/15	0.57
MLX 4-bit	9/21	5/15	0.54
GGUF f16	15/21	6/15	0.74

The reference run above (stock transformers fp32) scored 17/20 and 5/15. Quantizing to 8-bit changes little; MLX 4-bit loses the most (fewer replies end cleanly). The 21 prompts are the 20 evaluation prompts plus one follow-up question. The MLX columns detect a clean ending by length (an approximation), the GGUF columns by the real end-of-text marker, so compare formats within a family rather than across the two. Raw numbers: eval_results.json, eval_formats.json.

Limitations

Frequent factual mistakes and invented details; weak maths and logic; can loop on longer answers; 1,024-token context; English only; no safety tuning or refusal training, so do not deploy it unsupervised.

License

Apache-2.0 for the weights and code. Training data carries its own terms (see the dataset cards; Wikipedia and Stack Exchange are share-alike).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

VertexAGI

Model Tree

Base

VertexAGI/quartz-micro-preview-v2-base

Quantized

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Table with columns: Format, File / folder, Precision, Size, Works with
Format	File / folder	Precision	Size	Works with
Transformers (safetensors)	`model.safetensors` (repo root)	fp32 (32-bit)	400 MB	Hugging Face Transformers / PyTorch
MLX	`mlx/fp16/`	fp16 (16-bit)	200 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q4/`	4-bit (group size 64)	63 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q8/`	8-bit (group size 64)	113 MB	mlx-lm / MLX on Apple Silicon
GGUF	`gguf/quartz-micro-preview-v2-f16.gguf`	F16 (16-bit float)	201 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v2-q4_k_m.gguf`	Q4_K_M (4-bit)	67 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v2-q8_0.gguf`	Q8_0 (8-bit)	108 MB	llama.cpp, Ollama, LM Studio

Formats (one repo, three formats)

Table with columns: Where, Format
Where	Format
repo root	stock Hugging Face `LlamaForCausalLM` (fp32 safetensors), loads with plain `transformers` and plain `mlx_lm`
`mlx/fp16`, `mlx/q8`, `mlx/q4`	stock MLX (fp16, 8-bit, 4-bit)
`gguf/`	stock llama.cpp GGUF: `f16`, `q8_0`, `q4_k_m`

Every format was checked against the original training code on held-out text (perplexity; lower is better):

Table with columns: Format, Perplexity, Difference from original
Format	Perplexity	Difference from original
transformers fp32 (this repo root)	27.844	0.000%
MLX fp16	27.842	0.007%
MLX 8-bit	27.857	0.045%
MLX 4-bit	28.457	2.202%
GGUF f16	27.844	0.000%
GGUF Q8_0	27.850

Usage

python

# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v2")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v2")messages = [{"role": "user", "content": "Why do cats purr?"}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=400, do_sample=True, temperature=0.6, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

python

# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v2")print(generate(model, tok, prompt=tok.apply_chat_template([{"role": "user", "content": "Why do cats purr?"}], add_generation_prompt=True), max_tokens=400))

bash

# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v2 gguf/quartz-micro-preview-v2-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v2-q8_0.gguf -cnv

Instruction tuning

Base: Quartz Micro Preview V2 Base (VertexAGI/quartz-micro-preview-v2-base).
Data: 24,170 English conversations (9,258,981 tokens): filtered OpenAssistant OASST2 (reviewed, non-synthetic, top-ranked replies, follow-up turns kept) plus short conversational / rewriting / summarizing / constraint-following slices of smol-smoltalk. Long reasoning, maths, code and URLs were removed.
Method: full-parameter fine-tune, loss on assistant tokens only, 2 epochs, lr 1e-4 cosine, fp32.
Chat format: <|user|>\n{message}\n<|assistant|>\n{reply}<eos> (built into the tokenizer's chat template; no system role - put instructions in the user message).
Suggested sampling: temperature 0.6, top-p 0.9, repetition penalty 1.1.

Evaluation (hand-written prompts, not copied from the training sets)

20/20 prompts answered, 17/20 replies ended cleanly with <eos>, mean distinct-4-gram ratio 0.79.
Simple keyword fact checks: 5/15 (a deliberately easy test; misses are normal for this size).
Assistant-token loss on held-out chat data: base 2.856 -> tuned 2.280.
All prompts and replies are in eval_outputs.md, including the wrong ones. Machine-readable results: eval_results.json (per-prompt replies, gate checks) and eval_formats.json (all shipped formats).

Same prompts, every shipped format (greedy, repetition penalty 1.1, 400 new tokens)

Table with columns: Format, Ended with, Keyword fact checks, Distinct-4-gram
Format	Ended with	Keyword fact checks	Distinct-4-gram
MLX fp16	13/21	6/15	0.57
MLX 8-bit	12/21	6/15	0.57
MLX 4-bit	9/21	5/15	0.54
GGUF f16	15/21	6/15	0.74

Limitations

License

Apache-2.0 for the weights and code. Training data carries its own terms (see the dataset cards; Wikipedia and Stack Exchange are share-alike).