Lasimeri

GrugCap-27B

README

License: apache-2.0

Composition

Table with columns: Component, Source
Component	Source
`embed_tokens`, `lm_head`, final `norm`	ThinkingCap (identical in both parents)
Decoder blocks 0–9	ThinkingCap
Decoder blocks 10–44	grug-27b
Decoder blocks 45–63	ThinkingCap
Vision tower, MTP head	ThinkingCap
Tokenizer, chat template, config	ThinkingCap

463 of 1199 tensors come from grug: 24.8 GiB of 51.7 GiB, or 47.9% by weight. The 35 spliced blocks are 26 linear-attention and 9 full-attention layers, the hybrid pattern placing full attention every fourth layer.

Provenance was verified after the build by hashing tensors in the output against the same tensors fetched from both parent repos. The boundary is exact: blk.44.mlp.down_proj matches grug and not ThinkingCap, blk.45.mlp.down_proj matches ThinkingCap and not grug, with no unexplained tensors.

A useful incidental finding: the two parents have byte-identical embed_tokens, lm_head, every layernorm and every conv1d weight. They differ only in projection matrices, in all 64 blocks. That is the expected signature of grug's LoRA-on-linears method, and it means a provenance check that samples only layernorms cannot distinguish the parents at all.

What was tested

On a Q8_0 GGUF under llama.cpp on 2×RTX 3090:

Loads and generates cleanly; thinking mode intact and correctly delimited.
A small battery of reasoning prompts with deterministic answers: 4/4 on the auto-graded items, with the free-form items (bracket-matching code, strict JSON formatting, a 24-game expression, a deliberately unsatisfiable ordering puzzle) all correct on inspection.
220.6 tok/s prompt processing, 25.2 tok/s generation.

Thinking traces are terse and enumerative, closer to grug's register than to a verbose reasoner. Mean trace length across the battery was 155 characters. On the 24-game prompt it enumerated ten dead ends and verified the winner in 438 characters.

What was NOT tested

Be skeptical of this model until someone runs real evals. In particular:

No head-to-head comparison against either parent. No claim is made that this is better, or even as good.
No standard benchmarks (GSM8K, MMLU-Pro, GPQA, HumanEval). The battery above is a smoke test, not an evaluation.
No perplexity measurement.
No long-context, multi-turn, agentic, or tool-use testing.
Vision inputs untested, though the tower and projector are carried through.
No check for seam artifacts at the two splice boundaries, where a short healing finetune would be the obvious remedy if quality problems appear.

Files

BF16 safetensors in 11 shards, plus model-base-aux.safetensors holding ThinkingCap's MTP draft head. That aux file is not in the weight index; move it out of the directory before running convert_hf_to_gguf.py or the converter will trip over it.

Reproducing

bash

python splice_remote.py \  --recipient bottlecapai/ThinkingCap-Qwen3.6-27B \  --donor ProCreations/grug-27b \  --start 10 --end 45 \  --out ./GrugCap-27B

The splice is IO-bound, so re-cutting at different boundaries is cheap relative to any training. If you want a different band, change --start/--end; the tooling streams tensors by HTTP range request and never stores either parent locally.

Credits

bottlecapai/ThinkingCap-Qwen3.6-27B — host model, Apache-2.0
ProCreations/grug-27b — donor blocks, Apache-2.0
Qwen/Qwen3.6-27B — shared base

Neither parent's authors were involved in or endorse this merge.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Lasimeri

Model Tree

Base

bottlecapai/ThinkingCap-Qwen3.6-27B

Base

ProCreations/grug-27b

Merged

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality