ProCreations

grug-v2-9b

whole bird comparison

Same greedy harness, prompts, parser, runtime, and limits. HumanEval 164 tasks; MBPP first 100 sanitized test tasks; card 18 held-out actions; broad 119 held-out actions.

Table with columns: model, HumanEval, MBPP, card valid, card strict, card right, broad valid, broad strict, broad right
model	HumanEval	MBPP	card valid	card strict	card right	broad valid	broad strict	broad right
Ornith 1.0 9B	86.6	76.0	72.2	61.1	61.1	100.0	92.4	62.2
Grug v1 9B	78.7	76.0	88.9	88.9	88.9	99.2	86.6	91.6
Grug v2 before dialect fix	81.1	77.0	100.0	100.0	100.0	100.0	100.0	92.4
Grug v2 corrected	82.9	77.0	100.0	100.0	100.0	100.0	100.0	94.1

same-runtime rerun matter. old card number from different vLLM build not mixed into table. exact JSON rock included in results/.

grug family benchmark

Same prompts, parser, runtime, decoding, and limits for both birds. All numbers are percent; bold marks the best result in each column. Ties make both rocks bold.

Table with columns: model, HumanEval, MBPP, card valid, card strict, card right, broad valid, broad strict, broad right
model	HumanEval	MBPP	card valid	card strict	card right	broad valid	broad strict	broad right
Grug v2 9B	82.9	77.0	100.0	100.0	100.0	100.0	100.0	94.1
Grug 35B

dialect-fix capability gate

Table with columns: test, before dialect fix, after, change
test	before dialect fix	after	change
HumanEval pass@1 %	81.1	82.9	+1.8
MBPP pass@1 %	77.0	77.0	+0.0
card valid tool %	100.0	100.0	+0.0
card strict tool %	100.0	100.0	+0.0

valid = parser find offered tool call. strict = exact schema + required args. right tool = expected next action, not merely valid different club.

dialect gate

Separate 90-prompt held-out suite: 50 trivial, 20 moderate, 20 complex. No prompt used for gradient.

Table with columns: measure, before, after
measure	before	after
dialect-clean trace %	1.11	100.0
function-word ratio %	7.25	2.44
`User asks/wants/...` trace	89	0
`no tools needed` trace	70	0
`need to` trace

complex median gate protect brain meat: after must keep at least 80% old median and at least 25 word. grug remove grammar, not reasoning branch.

data repair

grug-think-v3-10k rewrites only private reasoning from v2 dataset.

10,000 trajectory / 62,722 think turn
11,889 changed trace
exact technical anchor retention: 100.0%
function-word ratio: 12.2% -> 11.2%
matched planner-English patterns after validation: 0
every visible answer, tool call, argument, result, system/user message unchanged

correction recipe

start public pre-fix ProCreations/grug-v2-9b
that checkpoint already carry verifier RL: valid XML, strict argument, right club, closed think reward with rope back to frozen Grug v1
rank-8 LoRA, alpha 16, LR 0.0001; final transformer layers 28 onward only
adapter delta scale base=0.75; self_attn=-0.470 selected by held-out style screen, then accepted only by whole capability gate
2,440 examples: {"calibration": 640, "coding": 805, "no_tool": 640, "tool": 355}
loss scope: private think tokens only; visible answer/tool call masked
max context 4,096; 1 epoch; vision tower frozen
synthetic calibration covers hello-world, direct multiplication, concise concept explanation
checkpoint release only after joint capability + dialect gate

pre-fix weights remain on branch pre-dialect-fix-2026-07-13.

brain + tool shape

Reasoning stays inside <think>...</think>. Native XML tool call stays:

xml
<tool_call>
<function=bash>
<parameter=command>
python -m pytest -q
</parameter>
</function>
</tool_call>

use

python
from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch

name = "ProCreations/grug-v2-9b"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForImageTextToText.from_pretrained(
    name, dtype=torch.bfloat16, device_map="auto")

Need Transformers with Qwen3.5 support. Popular GGUF rocks live at ProCreations/grug-v2-9b-gguf.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

ProCreations

Model Tree

Base

ProCreations/grug-9b

Fine-tuned

this model

Input Modalities

Text

Image

Video

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

whole bird comparison

Same greedy harness, prompts, parser, runtime, and limits. HumanEval 164 tasks; MBPP first 100 sanitized test tasks; card 18 held-out actions; broad 119 held-out actions.

Table with columns: model, HumanEval, MBPP, card valid, card strict, card right, broad valid, broad strict, broad right
model	HumanEval	MBPP	card valid	card strict	card right	broad valid	broad strict	broad right
Ornith 1.0 9B	86.6	76.0	72.2	61.1	61.1	100.0	92.4	62.2
Grug v1 9B	78.7	76.0	88.9	88.9	88.9	99.2	86.6	91.6
Grug v2 before dialect fix	81.1	77.0	100.0	100.0	100.0	100.0	100.0	92.4
Grug v2 corrected	82.9	77.0	100.0	100.0	100.0	100.0	100.0	94.1

same-runtime rerun matter. old card number from different vLLM build not mixed into table. exact JSON rock included in results/.

grug family benchmark

Same prompts, parser, runtime, decoding, and limits for both birds. All numbers are percent; bold marks the best result in each column. Ties make both rocks bold.

Table with columns: model, HumanEval, MBPP, card valid, card strict, card right, broad valid, broad strict, broad right
model	HumanEval	MBPP	card valid	card strict	card right	broad valid	broad strict	broad right
Grug v2 9B	82.9	77.0	100.0	100.0	100.0	100.0	100.0	94.1
Grug 35B

dialect-fix capability gate

Table with columns: test, before dialect fix, after, change
test	before dialect fix	after	change
HumanEval pass@1 %	81.1	82.9	+1.8
MBPP pass@1 %	77.0	77.0	+0.0
card valid tool %	100.0	100.0	+0.0
card strict tool %	100.0	100.0	+0.0

valid = parser find offered tool call. strict = exact schema + required args. right tool = expected next action, not merely valid different club.

dialect gate

Separate 90-prompt held-out suite: 50 trivial, 20 moderate, 20 complex. No prompt used for gradient.

Table with columns: measure, before, after
measure	before	after
dialect-clean trace %	1.11	100.0
function-word ratio %	7.25	2.44
`User asks/wants/...` trace	89	0
`no tools needed` trace	70	0
`need to` trace

complex median gate protect brain meat: after must keep at least 80% old median and at least 25 word. grug remove grammar, not reasoning branch.

data repair

grug-think-v3-10k rewrites only private reasoning from v2 dataset.

10,000 trajectory / 62,722 think turn
11,889 changed trace
exact technical anchor retention: 100.0%
function-word ratio: 12.2% -> 11.2%
matched planner-English patterns after validation: 0
every visible answer, tool call, argument, result, system/user message unchanged

correction recipe

start public pre-fix ProCreations/grug-v2-9b
that checkpoint already carry verifier RL: valid XML, strict argument, right club, closed think reward with rope back to frozen Grug v1
rank-8 LoRA, alpha 16, LR 0.0001; final transformer layers 28 onward only
adapter delta scale base=0.75; self_attn=-0.470 selected by held-out style screen, then accepted only by whole capability gate
2,440 examples: {"calibration": 640, "coding": 805, "no_tool": 640, "tool": 355}
loss scope: private think tokens only; visible answer/tool call masked
max context 4,096; 1 epoch; vision tower frozen
synthetic calibration covers hello-world, direct multiplication, concise concept explanation
checkpoint release only after joint capability + dialect gate

pre-fix weights remain on branch pre-dialect-fix-2026-07-13.

brain + tool shape

Reasoning stays inside <think>...</think>. Native XML tool call stays:

xml
<tool_call>
<function=bash>
<parameter=command>
python -m pytest -q
</parameter>
</function>
</tool_call>

use

python
from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch

name = "ProCreations/grug-v2-9b"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForImageTextToText.from_pretrained(
    name, dtype=torch.bfloat16, device_map="auto")

Need Transformers with Qwen3.5 support. Popular GGUF rocks live at ProCreations/grug-v2-9b-gguf.

grug-v2-9b

README

whole bird comparison

grug family benchmark

dialect-fix capability gate

dialect gate

data repair

correction recipe

brain + tool shape

use

Explore FriendliAI today

README

whole bird comparison

grug family benchmark

dialect-fix capability gate

dialect gate

data repair

correction recipe

brain + tool shape

use