Novasaki

Qwen3.5-2B-GrugSpeech-Q8

README

License: apache-2.0

🧠 Background: What is GPT-5.6 "Grug Speech"?

In GPT-5.6 reasoning traces, the model conducts its internal chain-of-thought using a terse, telegraphic syntax (e.g., "Need agent kind maybe open hands direct okay").

Instead of generating hundreds of polite, verbose conversational tokens ("In order to solve this problem, we must first consider..."), the model reasons with minimal semantic units:

Terse imperative verbs and nouns (Goal:, Check:, Fix:, Done.)
Direct causal operators (->, |, :)
Zero conversational fluff
Strict preservation of numbers, formulas, and causal logic

This allows agents to save massive token budgets across multi-turn reasoning loops.

🛠️ Model Architecture & Training

Base Model: Qwen/Qwen3.5-2B (hybrid linear + full attention architecture)
Quantization: 8-bit Quantized Training (BitsAndBytesConfig(load_in_8bit=True))
Adapter: LoRA (rank=16, alpha=32, target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
Dataset: Curated multi-domain reasoning subset (GSM8K arithmetic, AI2 ARC scientific reasoning, and complex agentic/systems engineering traces)
Training Convergence:
- : 0.5295

📊 Evaluation on Unseen Datasets

Tested on non-trained public reasoning benchmarks:

Table with columns: Benchmark / Domain, Input Tokens, Grug Tokens, Compression Ratio, Information Retention
Benchmark / Domain	Input Tokens	Grug Tokens	Compression Ratio	Information Retention
OpenBookQA (Biology / Science)	127	51	2.49x (-59.8%)	100% (Identified core constraint & correct choice)
MMLU (Elementary Mathematics)	149	61	2.44x (-59.1%)	100% (Algebraic step `24 / 2 = 12` & final answer preserved)
Systems Engineering (Web Crawler Optimization)	128

🚀 Quickstart & Inference

python

import torchfrom transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfigfrom peft import PeftModel base_model_id = "Qwen/Qwen3.5-2B"adapter_id = "Novasaki/Qwen3.5-2B-GrugSpeech-Q8" # 1. Load Tokenizertokenizer = AutoTokenizer.from_pretrained(adapter_id) # 2. Load Base Model in 8-bitbnb_config = BitsAndBytesConfig(load_in_8bit=True)base_model = AutoModelForCausalLM.from_pretrained(    base_model_id,    quantization_config=bnb_config,    device_map="auto") # 3. Attach LoRA Adaptermodel = PeftModel.from_pretrained(base_model, adapter_id)model.eval() # 4. Optimize Verbose Speech into Grug Speechsystem_prompt = (    "You are a Grug Speech Optimizer. Convert the verbose speech or reasoning into grug speech—an ultra-terse, "    "token-compressed internal reasoning style (GPT-5.6 grug brain). "    "Rules: Strip all conversational fluff, keep 100% of facts, numbers, logic steps, and conclusions.") verbose_input = """In order to optimize our distributed web crawler, we should replace our synchronous blocking HTTP client with an asynchronous event loop using uvloop and aiohttp. Currently, each crawling thread blocks waiting on socket I/O, which limits our throughput to roughly 50 requests per second and exhausts our system thread pool. By switching to an async connection pool with a maximum concurrency limit of 1000 open sockets and HTTP keep-alive enabled, we can achieve over 2,500 requests per second while reducing CPU utilization from 85% down to 25%.""" messages = [    {"role": "system", "content": system_prompt},    {"role": "user", "content": f"Optimize this verbose engineering speech into grug speech:\n\n{verbose_input}"}] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to("cuda") with torch.no_grad():    output_ids = model.generate(**inputs, max_new_tokens=150, do_sample=False) output = tokenizer.decode(output_ids[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(output)

Example Output:

text

Goal: optimize crawler throughput.Current bottleneck: blocking HTTP request per call -> 50 req/s, high CPU.Fix: switch to uvloop + aiohttp async client.Result: 2,500+ req/s, 75% lower CPU. Done.

Created by Novasaki

🏆 Full-Suit Master Leaderboard: 11 Models Under Strict Single-Model Isolation

Evaluated across 7 multi-lingual and multi-complexity scenarios on the live enterprise Data Analysis Agent (sales.xlsx, 100,300 rows) with 100% VRAM purge between runs:

Table with columns: Model Name, Architecture Family, Quant Format, Memory (GB), Pass Rate, Avg Wall (s), Gen Speed (t/s), Prefill (t/s), Avg CoT (tok), Schema OK
Model Name	Architecture Family	Quant Format	Memory (GB)	Pass Rate	Avg Wall (s)	Gen Speed (t/s)	Prefill (t/s)	Avg CoT (tok)	Schema OK
`bonsai-8b-prismml`	PrismML 1-Bit	`Q1_0`	1.08 GB	6/7 (85.7%)	1.16s

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Novasaki

Model Tree

Base

Qwen/Qwen3.5-2B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality