VertexAGI

quartz-micro-preview-v3-base

README

License: apache-2.0

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Table with columns: Format, File / folder, Precision, Size, Works with
Format	File / folder	Precision	Size	Works with
Transformers (safetensors)	`model.safetensors` (repo root)	fp32 (32-bit)	400 MB	Hugging Face Transformers / PyTorch
MLX	`mlx/fp16/`	fp16 (16-bit)	200 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q4/`	4-bit (group size 64)	63 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q8/`	8-bit (group size 64)	113 MB	mlx-lm / MLX on Apple Silicon
GGUF	`gguf/quartz-micro-preview-v3-base-f16.gguf`	F16 (16-bit float)	201 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v3-base-q4_k_m.gguf`	Q4_K_M (4-bit)	67 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v3-base-q8_0.gguf`	Q8_0 (8-bit)	108 MB	llama.cpp, Ollama, LM Studio

Formats (one repo, three formats)

Table with columns: Where, Format
Where	Format
repo root	stock Hugging Face `LlamaForCausalLM` (fp32 safetensors), loads with plain `transformers` and plain `mlx_lm`
`mlx/fp16`, `mlx/q8`, `mlx/q4`	stock MLX (fp16, 8-bit, 4-bit)
`gguf/`	stock llama.cpp GGUF: `f16`, `q8_0`, `q4_k_m`

Every format was checked against the original training code on held-out text (perplexity; lower is better):

Table with columns: Format, Perplexity, Difference from original
Format	Perplexity	Difference from original
transformers fp32 (this repo root)	10.614	0.000%
MLX fp16	10.614	0.000%
MLX 8-bit	10.604	0.096%
MLX 4-bit	11.043	4.044%
GGUF f16	10.614	0.002%
GGUF Q8_0	10.615	0.008%
GGUF Q4_K_M	10.903	2.717%

Usage

python

# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v3-base")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v3-base")ids = tok("The history of the Roman Empire", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0], skip_special_tokens=True))

python

# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v3-base")print(generate(model, tok, prompt="The history of the Roman Empire", max_tokens=60))

bash

# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v3-base gguf/quartz-micro-preview-v3-base-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v3-base-q8_0.gguf -p "The history of the Roman Empire" -n 60 -no-cnv

Training

Data: | Source | Unique tokens | Notes | |---|---:|---| | FineWeb-Edu (dedup, SmolLM-Corpus) | 2.60B | fills the rest of the budget | | Cosmopedia v2 (SmolLM-Corpus) | 1.30B | synthetic textbook-style text | | Wikipedia (20231101.en) | 0.78B | | | DCLM-Baseline | 0.52B | | | Stack Exchange (preferences dump, questions + answers) | 0.46B | | | FineMath (4+) | 0.39B | | | Python-Edu (cleaned) | 0.33B | | | Chat / instruction text (SmolTalk subsets, Tulu-3 SFT mixture subsets) | 0.07B | chat-format documents, includes "I can't do that" / abstention examples | | ELI5 | 30M | question/answer pairs | | SQuAD 2.0 (answerable 9.6M, unanswerable 10.1M x3 passes) | 20M + repeats | the "unanswerable" ones teach "the passage does not say" | | TriviaQA (no-context) | 3.2M x2 | |

Total about 6.5B training tokens (6,513,293,092 with the repeats), one pass, 99,384 steps of 65,536 tokens. 10-word-window n-gram decontamination was run against ARC, HellaSwag, PIQA, WinoGrande, LAMBADA, OpenBookQA, SciQ, BoolQ, TriviaQA-validation, MMLU-test and CommonsenseQA text before packing.

Schedule: AdamW (betas 0.9 / 0.95), lr 6e-4 with 300 warm-up steps then cosine to 6e-5, micro-batch 2 x 32 accumulation x 1,024 tokens, fp32 on one GTX 1660 Ti (about 18 days at ~4,200 tokens/s).
Final validation loss: 2.61 (perplexity 13.6) on the V3 validation split.
Versus the V2 base (same architecture and tokenizer, 100M training tokens): on 300 random 1,024-token windows of the V3 validation split, perplexity is 14.4 (V3 base) vs 28.1 (V2 base). The validation text is drawn from the V3 mix, so the V2 base is partly out of distribution; read it as "V3 has seen far more, and more varied, text", not as a like-for-like quality gap.

Limitations

No instruction following, no safety tuning, English only, 1,024-token context, and it inherits the errors and biases of web and synthetic text. Not suitable for any decision that matters.

Data and licences (read before commercial use)

The weights are released under Apache-2.0, but they were trained on third-party data with its own terms, and some of those terms are NonCommercial or ShareAlike (for example the Tulu-3 mixture includes no_robots, CC-BY-NC; Wikipedia, Stack Exchange, Dolly and SQuAD are CC-BY-SA; CoCoNot is ODC-By). Whether training-data terms extend to model weights is legally unsettled. This is a research / hobby model: if you need legal certainty for a commercial product, ask a lawyer or retrain on data you have cleared. The pretraining and identity datasets are not public.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

VertexAGI

Model Tree

Base

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Table with columns: Format, File / folder, Precision, Size, Works with
Format	File / folder	Precision	Size	Works with
Transformers (safetensors)	`model.safetensors` (repo root)	fp32 (32-bit)	400 MB	Hugging Face Transformers / PyTorch
MLX	`mlx/fp16/`	fp16 (16-bit)	200 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q4/`	4-bit (group size 64)	63 MB	mlx-lm / MLX on Apple Silicon
MLX	`mlx/q8/`	8-bit (group size 64)	113 MB	mlx-lm / MLX on Apple Silicon
GGUF	`gguf/quartz-micro-preview-v3-base-f16.gguf`	F16 (16-bit float)	201 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v3-base-q4_k_m.gguf`	Q4_K_M (4-bit)	67 MB	llama.cpp, Ollama, LM Studio
GGUF	`gguf/quartz-micro-preview-v3-base-q8_0.gguf`	Q8_0 (8-bit)	108 MB	llama.cpp, Ollama, LM Studio

Formats (one repo, three formats)

Table with columns: Where, Format
Where	Format
repo root	stock Hugging Face `LlamaForCausalLM` (fp32 safetensors), loads with plain `transformers` and plain `mlx_lm`
`mlx/fp16`, `mlx/q8`, `mlx/q4`	stock MLX (fp16, 8-bit, 4-bit)
`gguf/`	stock llama.cpp GGUF: `f16`, `q8_0`, `q4_k_m`

Every format was checked against the original training code on held-out text (perplexity; lower is better):

Table with columns: Format, Perplexity, Difference from original
Format	Perplexity	Difference from original
transformers fp32 (this repo root)	10.614	0.000%
MLX fp16	10.614	0.000%
MLX 8-bit	10.604	0.096%
MLX 4-bit	11.043	4.044%
GGUF f16	10.614	0.002%
GGUF Q8_0	10.615	0.008%
GGUF Q4_K_M	10.903	2.717%

Usage

python

# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v3-base")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v3-base")ids = tok("The history of the Roman Empire", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0], skip_special_tokens=True))

# stock transformers (no custom code, no trust_remote_code)from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v3-base")model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v3-base")ids = tok("The history of the Roman Empire", return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.1)print(tok.decode(out[0], skip_special_tokens=True))

python

# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v3-base")print(generate(model, tok, prompt="The history of the Roman Empire", max_tokens=60))

# stock MLX (Apple silicon)from mlx_lm import load, generatemodel, tok = load("VertexAGI/quartz-micro-preview-v3-base")print(generate(model, tok, prompt="The history of the Roman Empire", max_tokens=60))

bash

# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v3-base gguf/quartz-micro-preview-v3-base-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v3-base-q8_0.gguf -p "The history of the Roman Empire" -n 60 -no-cnv

# stock llama.cpp (GGUF in the gguf/ folder)hf download VertexAGI/quartz-micro-preview-v3-base gguf/quartz-micro-preview-v3-base-q8_0.gguf --local-dir .llama-cli -m gguf/quartz-micro-preview-v3-base-q8_0.gguf -p "The history of the Roman Empire" -n 60 -no-cnv

Training

Data: | Source | Unique tokens | Notes | |---|---:|---| | FineWeb-Edu (dedup, SmolLM-Corpus) | 2.60B | fills the rest of the budget | | Cosmopedia v2 (SmolLM-Corpus) | 1.30B | synthetic textbook-style text | | Wikipedia (20231101.en) | 0.78B | | | DCLM-Baseline | 0.52B | | | Stack Exchange (preferences dump, questions + answers) | 0.46B | | | FineMath (4+) | 0.39B | | | Python-Edu (cleaned) | 0.33B | | | Chat / instruction text (SmolTalk subsets, Tulu-3 SFT mixture subsets) | 0.07B | chat-format documents, includes "I can't do that" / abstention examples | | ELI5 | 30M | question/answer pairs | | SQuAD 2.0 (answerable 9.6M, unanswerable 10.1M x3 passes) | 20M + repeats | the "unanswerable" ones teach "the passage does not say" | | TriviaQA (no-context) | 3.2M x2 | |

Schedule: AdamW (betas 0.9 / 0.95), lr 6e-4 with 300 warm-up steps then cosine to 6e-5, micro-batch 2 x 32 accumulation x 1,024 tokens, fp32 on one GTX 1660 Ti (about 18 days at ~4,200 tokens/s).
Final validation loss: 2.61 (perplexity 13.6) on the V3 validation split.
Versus the V2 base (same architecture and tokenizer, 100M training tokens): on 300 random 1,024-token windows of the V3 validation split, perplexity is 14.4 (V3 base) vs 28.1 (V2 base). The validation text is drawn from the V3 mix, so the V2 base is partly out of distribution; read it as "V3 has seen far more, and more varied, text", not as a like-for-like quality gap.

Limitations

No instruction following, no safety tuning, English only, 1,024-token context, and it inherits the errors and biases of web and synthetic text. Not suitable for any decision that matters.