saidutta69

quark-v2

README

License: apache-2.0

Model Details

Table with columns: Property, Value
Property	Value
Base Model	openbmb/MiniCPM5-1B
Architecture	LlamaForCausalLM (24 layers, 16 heads, 2 KV heads, 1536 hidden)
Parameters	1.0B
Precision	BF16 (training), Q8_0 (GGUF inference)
Context Length	131,072 tokens
Vocabulary	130,560 (BPE)
Training	Full BF16 SFT via TRL `SFTTrainer`
Training Hardware	NVIDIA RTX PRO 6000 Blackwell (96 GB VRAM)
Training Duration	~58 min (5 epochs, 22.4K train / 2.5K eval)
Dataset	24,909 quality-filtered examples (worker 8,217 + judge 10,856 + planner 5,836)
Train Loss	0.90
Eval Loss	0.84

What It Does

QuarkV2 plays three roles via system prompt switching:

Worker — Routes queries to tools (web search, page reader, GitHub API) using structured XML function calls
Judge — Produces binary REACHED / NOT_REACHED verdicts with concise reasoning
Planner — Generates step-by-step research plans with tool-call sequences

Compared to the base model (generic chat), V2 produces focused, structured output with correct tool-call XML. Compared to V1 (too terse, ~40 tok avg), V2 balances brevity with completeness (~57 tok avg).

Inference (GGUF)

Recommended for CPU inference via llama-cpp-python:

python
from llama_cpp import Llama

llm = Llama(
    model_path="quark-v2-q8_0.gguf",
    n_ctx=32768,
    n_threads=8,
    verbose=False,
)

output = llm(
    "<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n",
    max_tokens=128,
    temperature=0,
    stop=["<|im_end|>"],
)
print(output["choices"][0]["text"])

Chat Template

The model uses the MiniCPM chat template with |im_start| / |im_end| format and tool-use XML:

markdown
<|im_start|>system
You are a research assistant.
<|im_end|>
<|im_start|>user
Query here
<|im_end|>
<|im_start|>assistant

Training Details

Dataset

Generated using the gpt-oss-120b model for focused synthetic data generation. 24,909 examples across three roles, filtered for quality and deduplicated via self-consistency scoring.

Table with columns: Role, Train, Eval, Total
Role	Train	Eval	Total
Worker	7,395	822	8,217
Judge	9,770	1,086	10,856
Planner	5,252	584	5,836
Total	22,417	2,492	24,909

Hyperparameters

Table with columns: Parameter, Value
Parameter	Value
Learning Rate	2e-5
Batch Size	16 × 2 gradient accumulation
Scheduler	Cosine, 5% warmup
Epochs	5
Precision	BF16
Max Seq Length	2048

Files

model.safetensors — Full BF16 weights (2.1 GB)
quark-v2-q8_0.gguf — Q8_0 quantized GGUF for CPU inference (1.15 GB)

License

Apache 2.0

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

saidutta69

Model Tree

Base

openbmb/MiniCPM5-1B

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Details

Table with columns: Property, Value
Property	Value
Base Model	openbmb/MiniCPM5-1B
Architecture	LlamaForCausalLM (24 layers, 16 heads, 2 KV heads, 1536 hidden)
Parameters	1.0B
Precision	BF16 (training), Q8_0 (GGUF inference)
Context Length	131,072 tokens
Vocabulary	130,560 (BPE)
Training	Full BF16 SFT via TRL `SFTTrainer`
Training Hardware	NVIDIA RTX PRO 6000 Blackwell (96 GB VRAM)
Training Duration	~58 min (5 epochs, 22.4K train / 2.5K eval)
Dataset	24,909 quality-filtered examples (worker 8,217 + judge 10,856 + planner 5,836)
Train Loss	0.90
Eval Loss	0.84

What It Does

QuarkV2 plays three roles via system prompt switching:

Worker — Routes queries to tools (web search, page reader, GitHub API) using structured XML function calls
Judge — Produces binary REACHED / NOT_REACHED verdicts with concise reasoning
Planner — Generates step-by-step research plans with tool-call sequences

Inference (GGUF)

Recommended for CPU inference via llama-cpp-python:

python
from llama_cpp import Llama

llm = Llama(
    model_path="quark-v2-q8_0.gguf",
    n_ctx=32768,
    n_threads=8,
    verbose=False,
)

output = llm(
    "<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n",
    max_tokens=128,
    temperature=0,
    stop=["<|im_end|>"],
)
print(output["choices"][0]["text"])

Chat Template

The model uses the MiniCPM chat template with |im_start| / |im_end| format and tool-use XML:

markdown
<|im_start|>system
You are a research assistant.
<|im_end|>
<|im_start|>user
Query here
<|im_end|>
<|im_start|>assistant

Training Details

Dataset

Generated using the gpt-oss-120b model for focused synthetic data generation. 24,909 examples across three roles, filtered for quality and deduplicated via self-consistency scoring.

Table with columns: Role, Train, Eval, Total
Role	Train	Eval	Total
Worker	7,395	822	8,217
Judge	9,770	1,086	10,856
Planner	5,252	584	5,836
Total	22,417	2,492	24,909

Hyperparameters

Table with columns: Parameter, Value
Parameter	Value
Learning Rate	2e-5
Batch Size	16 × 2 gradient accumulation
Scheduler	Cosine, 5% warmup
Epochs	5
Precision	BF16
Max Seq Length	2048

Files

model.safetensors — Full BF16 weights (2.1 GB)
quark-v2-q8_0.gguf — Q8_0 quantized GGUF for CPU inference (1.15 GB)

License

Apache 2.0