Roswaal-8B API & Inference Endpoint

Benchmark Results

Evaluated on the full GSM8K test set (1,319 problems) using lm-evaluation-harness with a 5-shot prompt configuration (temperature=0.1 / low-temp reasoning).

Table with columns: Model, GSM8K Accuracy
Model	GSM8K Accuracy
Gemma 3 4B IT	89.2%
Qwen2.5 Coder 14B Instruct	88.7%
Phi-4-mini	88.6%
Roswaal-8B	87.64%
Qwen2.5 Coder 7B Instruct	86.7%
Phi 3.5 Mini Instruct	86.2%
Phi-3 Medium (4k-instruct)	85.2%
Gemma 2 9B	84.9%
Llama 3.1 8B Instruct	82.4%
Qwen3-8B	79.4%
Mistral-7B	77.9%
Llama 3.2 3B	77.7%

Roswaal-8B scores 8.24 p.p. above Qwen3-8B on GSM8K.

Methodology: Why It Works

Unlike standard fine-tuning processes that attempt to map a question directly to an answer, Roswaal-8B was explicitly trained on ~20,000 highly curated Chain-of-Thought (CoT) sequences.

The training objective forces the model to:

Deconstruct complex prompts into smaller, actionable logical steps.
Self-correct during the generation phase (e.g., catching internal arithmetic errors before outputting the final answer).
Strictly isolate its internal monologue from the user-facing output using specialized structural tags.

Hyperparameters

Table with columns: Parameter, SFT
Parameter	SFT
Method	LoRA (16-bit)
LoRA rank (r)	16
LoRA alpha	32
LoRA targets	"q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"
Learning rate	1e-4
Scheduler	Cosine
Optimizer	adamw_8bit
Epochs	1
Batch size	8

Prompt Format & Generation Strategy

Roswaal-8B relies on the standard ChatML template but requires a specific generation logic. The model expects to enclose its reasoning process inside <think>...</think> tags.

Recommended Generation Parameters:

Temperature: 0.1 to 0.6 (Keep it low to prevent logical drift during complex math).
Top_p: 0.9
Max_new_tokens: 1024 - 4096 (Crucial: The model needs enough token space to "think" before answering. Do not restrict this too heavily).

Developed by maxzt

Benchmark Results

Evaluated on the full GSM8K test set (1,319 problems) using lm-evaluation-harness with a 5-shot prompt configuration (temperature=0.1 / low-temp reasoning).

Table with columns: Model, GSM8K Accuracy
Model	GSM8K Accuracy
Gemma 3 4B IT	89.2%
Qwen2.5 Coder 14B Instruct	88.7%
Phi-4-mini	88.6%
Roswaal-8B	87.64%
Qwen2.5 Coder 7B Instruct	86.7%
Phi 3.5 Mini Instruct	86.2%
Phi-3 Medium (4k-instruct)	85.2%
Gemma 2 9B	84.9%
Llama 3.1 8B Instruct	82.4%
Qwen3-8B	79.4%
Mistral-7B	77.9%
Llama 3.2 3B	77.7%

Roswaal-8B scores 8.24 p.p. above Qwen3-8B on GSM8K.

Methodology: Why It Works

Unlike standard fine-tuning processes that attempt to map a question directly to an answer, Roswaal-8B was explicitly trained on ~20,000 highly curated Chain-of-Thought (CoT) sequences.

The training objective forces the model to:

Deconstruct complex prompts into smaller, actionable logical steps.
Self-correct during the generation phase (e.g., catching internal arithmetic errors before outputting the final answer).
Strictly isolate its internal monologue from the user-facing output using specialized structural tags.

Hyperparameters

Table with columns: Parameter, SFT
Parameter	SFT
Method	LoRA (16-bit)
LoRA rank (r)	16
LoRA alpha	32
LoRA targets	"q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"
Learning rate	1e-4
Scheduler	Cosine
Optimizer	adamw_8bit
Epochs	1
Batch size	8

Prompt Format & Generation Strategy

Roswaal-8B relies on the standard ChatML template but requires a specific generation logic. The model expects to enclose its reasoning process inside <think>...</think> tags.

Recommended Generation Parameters:

Temperature: 0.1 to 0.6 (Keep it low to prevent logical drift during complex math).
Top_p: 0.9
Max_new_tokens: 1024 - 4096 (Crucial: The model needs enough token space to "think" before answering. Do not restrict this too heavily).

Developed by maxzt

Roswaal-8B

README

Benchmark Results

Methodology: Why It Works

Hyperparameters

Prompt Format & Generation Strategy

Explore FriendliAI today

README

Benchmark Results

Methodology: Why It Works

Hyperparameters

Prompt Format & Generation Strategy