What Changed
The training target is a terse internal-reasoning style: short high-density
steps, fewer filler words, and explicit preservation of key constraints,
branching decisions, invariants, edge cases, and final-answer checks.
The goal is lower reasoning-token usage relative to the base model while
preserving answer quality. It is not meant to hide uncertainty or remove needed
reasoning.
Training Data
The data pipeline started from a recent, filtered reasoning pool and converted
verbose traces into compact traces before SFT packing.
Source gate:
- Run date: June 30, 2026.
- Default freshness cutoff: 45 days. Sources older than May 16, 2026 were
rejected unless manually allowed.
- Allowed train licenses: MIT, Apache-2.0, CC-BY-4.0, CC0-1.0.
- Hard reject terms included OpenAI, ChatGPT, GPT-5, Claude, Anthropic, Opus,
Sonnet, and Gemini.
- Soft-risk sources marked as synthetic/distill were manually reviewed or
rejected depending on provenance and license.
Final verified source mix:
Table with columns: Source, License, Domain, Verified rows| Source | License | Domain | Verified rows |
|---|
hotdogs/uka-glm-5.2 | MIT | agent code | 1,617 |
Scale-or-Reason/general-reasoning-ift-pairs | MIT | general reasoning | 1,305 |
samcheng0/lumia-reasoning-sft-v1 | Apache-2.0 | code reasoning | 1,103 |
HSH-Intelligence/verified-math-reasoning-3k | Apache-2.0 | math | 672 |
|
Row counts:
- Normalized recent reasoning pool: 8,680 rows.
- Selected verbose reasoning set: 6,144 rows.
- Compact raw transform output: 6,144 rows.
- Verified compact rows: 5,740 rows.
- Rejected compact rows: 404 rows.
- Packed SFT split: 5,166 train / 287 validation / 287 test.
The compact reasoning transform was generated with
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit served by vLLM. Rows were checked for
compression ratio, answer preservation, and obvious loss of critical reasoning
information before training.
Training Procedure
Training was completion-only SFT: prompt tokens were masked with -100, and
only the assistant completion was trained.
Core settings:
- Base model:
google/gemma-4-12B-it.
- Method: QLoRA / PEFT LoRA, merged into full model weights for upload.
- Quantization during training: 4-bit NF4 with BF16 compute.
- Max sequence length: 6,144.
- LoRA rank: 16.
- LoRA alpha: 32.
- LoRA dropout: 0.05.
- Target modules:
q_proj, k_proj, v_proj, o_proj, gate_proj,
up_proj, down_proj.
- Batch size: 1.
No train or validation rows were skipped in the final run.
Local Evaluation
Small local EOS-only math proxy eval, no generation token cap:
Table with columns: Model, Rows, Total generated tokens, Avg generated tokens, Proxy accuracy, Numeric last-match rate| Model | Rows | Total generated tokens | Avg generated tokens | Proxy accuracy | Numeric last-match rate |
|---|
google/gemma-4-12B-it base | 36 | 8,227 | 228.53 | 91.7% | 86.1% |
| Grug 12B | 36 | 2,482 | 68.94 | 100.0% | 100.0% |
This is a small proxy eval, not a broad benchmark. Treat it as a smoke test
showing the intended token-efficiency direction, then run your own benchmark.
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "kai-os/Grug-12B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
model.eval()
messages = [
{"role": "user", "content": "If a shirt is $80 and goes 25% off, what is the sale price?"}
]
inputs = tokenizer.apply_chat_template(
messages,
return_tensors="pt",
add_generation_prompt=True,
).to(model.device)
with torch.no_grad():
output = model.generate(inputs, do_sample=False, max_new_tokens=512)
print(tokenizer.decode(output[0], skip_special_tokens=True))
For token-efficiency tests, compare against the base model with the same prompt,
same decoding settings, and no artificial token cap unless your deployment
requires one.
Limitations
- This is an experimental fine-tune.
- It may over-compress reasoning on tasks that need longer derivations.
- It inherits the base model's limitations and safety behavior.
- The reported eval is small and local.
- The dataset includes synthetic and distilled reasoning traces from the
listed open datasets; review source licenses and provenance before using this
in commercial or sensitive settings.
Acknowledgements
Thanks to Lambda, the inference provider, for compute
credits that supported the dataset work, training, and evaluation.