exnivo

Qwen3.8-20B-Minitron

README

License: apache-2.0

Architecture

Table with columns: Qwen3.8-27B, This model
	Qwen3.8-27B	This model
Measured parameters	27.357B	19.746B
Language layers	64	44
Hybrid groups	16	11
Parameter reduction	—	27.82%

Approximately 7.61 billion parameters were physically removed.

Original language-layer pattern:

[linear_attention, linear_attention, linear_attention, full_attention] × 16

Surviving groups:

[0, 2, 4, 6, 7, 8, 11, 12, 13, 14, 15]

Removed groups:

[1, 3, 5, 9, 10]

The surviving layers retain their original Qwen weights and are reindexed after pruning.

Recovery

The raw 19.746B child remained coherent but showed noticeable reasoning and factual instability.

Recovery was performed in multiple stages using:

original Qwen3.8-27B teacher logits
top-k knowledge distillation
next-token training
mixed educational/instruction/math/code data
targeted repair examples
custom LoRA recovery
final LoRA merge into the standalone weights

The uploaded checkpoint contains the fully merged weights. No adapter or pruning script is required to load it.

Internal release checks

A custom held-out evaluation used during release selection scored:

overall accuracy: 81.2%
arithmetic: 100%
logic: 100%
money/Dutch: 83%
probability: 80%
Python: 75%
strict instruction following: 83%
sequence/pattern reasoning: 20%

These are custom internal checks, not standardized benchmark scores, and should not be compared directly with official Qwen benchmark results.

The final legacy regression suite scored 11/12 (91.7%).

Known limitations

This is an experimental compressed model.

In particular:

pattern/sequence reasoning remains a weakness
some unusual wording can still produce incorrect reasoning
pruning may have removed capabilities not represented by our tests
recovery and evaluation focused primarily on text
the vision tower is retained from the parent model but was not the focus of the recovery process
this model should not be assumed to match Qwen3.8-27B quality

Usage

python
import torch
from transformers import AutoModelForMultimodalLM, AutoTokenizer

model_id = "exnivo/Qwen3.8-20B-Minitron"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "What is the capital of Australia?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(
    text,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=128,
    )

print(
    tokenizer.decode(
        output[0][inputs.input_ids.shape[1]:],
        skip_special_tokens=True,
    )
)

Method

The pruning process did not rank all layers once and then remove them simultaneously.

Instead, after every group removal, candidate importance was measured again on the current pruned model using marginal KL divergence.

This mattered because layer importance changed substantially after earlier groups were removed.

The process was:

start with 16 four-layer hybrid groups
protect boundary groups
temporarily bypass each candidate group
measure marginal KL divergence
permanently remove the least disruptive group
recompute all candidate scores
repeat until 11 groups remained
recover the resulting model using distillation
run targeted repair training
merge recovery weights into this standalone checkpoint

Attribution

Based on Qwen/Qwen3.8-27B.

This is an independent experimental derivative and is not an official Qwen release.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider