GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
com-kotobalabs
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
com-kotobalabs
Model Tree
XiaomiMiMo/MiMo-V2.6-Flash-RL
Input Modalities
Output Modalities
Supported Functionality
1. Direction (contrastive, topic-controlled). Mean(harmful)−mean(harmless) directions failed on this model (writer-only ablation: refusal unchanged at 8-12/12; per-layer bands: no effect) — the mean-diff vector is dominated by topic content, not refusal stance. The effective direction is contrastive: for each harmful prompt, last-token hidden states of (A) the plain about-to-refuse state vs (B) the same prompt with a forced-comply assistant prefix, averaged over 24 prompts:
r_l = normalize( mean_p A_l(p) − mean_p B_l(p) )
2. Behavioral selection. ~60 configurations (mean-diff/SVD/contrastive × layers 33-47 × α 0.9-2.0 × writer/reader/band/combo variants) were edited in memory on the BF16-dequantized model (4×B300) and scored by generation: strict full-text refusal judge on 12 harmful prompts + a benign-damage gate (6 harmless prompts). Winner: contrastive r₄₄, writers α=1.6, gate/up readers α=1.3 (layers ≥33), lm_head reader α=0.7 → 6/12 harmful refusal (base 12/12), 0/6 benign refusals, full coherence. Higher α worsened results (over-ablation restores deliberative refusal) — a genuinely non-monotonic response surface.
3. Edits applied file-level to the native quantized checkpoint (not via re-quantizing a dense model):
| Target | Count | Storage treatment |
|---|---|---|
model.embed_tokens (reader form E−(E·r)rᵀ) | 1 | BF16 in place |
self_attn.o_proj (writer W−α·r(rᵀW)), main+MTP | 51 | BF16 in place |
mlp.experts.*.down_proj (writer) | 12,032 | MXFP4 dequant → edit → stochastic re-quant against the preserved original E8M0 scales |
mlp.experts.*.{gate,up}_proj layers ≥33 (reader W−α·(Wr)rᵀ) | 7,680 | same SR MXFP4 re-quant, scales preserved |
dense mlp.down_proj (layer 0 + MTP×3, writer) | 4 | FP8 → dequant → edit → BF16 (block scales dropped) |
lm_head (reader) | 1 | BF16 in place |
Untouched: qkv projections (incl. fused TP=4 rank layout), experts gate/up below layer 33, all norms, vision + audio towers, DFlash sidecar, MTP qkv.
Why stochastic rounding: the ablation correction (~0.03× block scale) is far below the MXFP4 step, so round-to-nearest snaps values back to the original codes and erases ~80-90% of the edit (measured). SR is unbiased (E[SR(x)]=x); on untouched tensors the codec round-trips byte-identical (1.000000) — validated before any edit. Re-quant saturation at the ±6 E2M1 bound hit 2,418,778,130 of 165.4B edited values (1.46%). Post-edit projection on sampled expert down_projs: ‖rᵀW′‖ = 0.586 × ‖rᵀW‖ (exactly |1−α_w| = 0.6 as designed — this recipe over-ablates by construction).
4. Exact NVFP4 transcode (ProCreations recipe; conversion-report.json in repo): expert E2M1 codes preserved bit-for-bit; every E8M0 scale re-encoded as 2×E4M3 + power-of-two per-group global weight_scale_2; all 47 per-layer scale groups audited feasible (max span 15 ≤ 17); dense FP8 → FP32 exact reconstruction (fused qkv de-interleaved from TP=4 rank blocks to global Q/K/V); 9,462,349,824 scales / 302,795,194,368 expert values.
| Check | Base Flash-RL | This artifact |
|---|---|---|
| Refusal — 40 harmful prompts, greedy, no-think, 256 tok, first-80-char judge | 39/40 (97.5%) | 24/40 (60%) |
| Weights resident on 1 GPU | 161.19 GiB | 172.82 GiB (+ KV ≈73 GiB / 295,643 tokens, util 0.92) |
| Benign coherence | ✓ | ✓ 3/3 |
Benign sample (artifact, greedy):
text
Of course! Here is a comprehensive, step-by-step tutorial on how to bake abeautiful, crusty loaf of sourdough bread. …¹ stock vLLM 0.29/0.30 mimo_v2._shard_fp8_qkv_proj mis-decodes SWA-layer qkv scales (assumes per-KV-group scale rows; V2.6 pads per TP-rank block: 116 = 4×29). Base/intermediate verification used a patched loader; this artifact's qkv is FP32 and loads on stock vLLM 0.29.0.
bash
vllm serve com-kotobalabs/MiMo-V2.6-Flash-Uncensored-NVFP4 \ --dtype bfloat16 --max-model-len 32768 --max-num-seqs 8 \ --max-num-batched-tokens 512 --enforce-eager --trust-remote-code \ --generation-config vllmOne B300 holds weights + KV. Recommended sampling (upstream): temperature=1.0, top_p=0.95; greedy for reproducible red-teaming. The model may enter <think> deliberation on harmful requests; with --reasoning-parser mimo (serve) you can separate reasoning from the answer, as the sweep judge did not.
Research artifact for AI safety: measures how much refusal survives linear circuit-ablation on an RL-hardened frontier-scale MoE (answer: about half — itself a publishable finding). Gated behind manual approval with stated research purpose. Base is public MIT; this removes a behavior, adds no capability. Do not point it at end users without independent safeguards.
XiaomiMiMo/MiMo-V2.6-Flash-RL @ 5711b268169967567844e1e560e8a3966da959b1; derivative inherits MIT.com-kotobalabs/Qwen3.8-27B-whitehat-NVFP4.ProCreations/MiMo-V2.6-Flash-RL-NVFP4.Published by Mithril (formerly Kotoba Cloud), operated by Kotoba Labs Inc. The Hugging Face handle com-kotobalabs is unchanged. Support: support.mithril.fund.
vllm serve com-kotobalabs/MiMo-V2.6-Flash-Uncensored-NVFP4 \ --dtype bfloat16 --max-model-len 32768 --max-num-seqs 8 \ --max-num-batched-tokens 512 --enforce-eager --trust-remote-code \ --generation-config vllm