| Base model | Qwen/Qwen3.5-35B-A3B (254-Expert Sovereign MoE Architecture, top_k=8, 40 layers) |
| Previous public release | kataguru/Kataguru-Sceptic-35B-A3B-v2.8.1-NVFP4 (Σ15=69.24 %) |
| Kataguru v3.1 modifications | 2-Cycle Orthonormal Subspace MoE Training (Sykli 1 cp-400 + Sykli 2 cp-500): Cycle 1 (qr_frozen_a orthonormal subspace + 4-Level Orthogonal Subspace Protection against v2.8.1) on 100 % cognitive, Finnish, graduate STEM, clinical medicine & EPO/PRH patent data; Cycle 2 (weakest_link_oplora + Cross-Mixture 64/36 Pohjantähti Anchor Shield + Live Checkpoint Epistemic Oracle pruning NLL < 0.35 samples) on PASSED_EXEC-verified competitive algorithms, multi-turn SWE, BFCL tool calling, and industrial PLC/CNC engineering |
| Key capability gains vs. v2.8.1 | GPQA Diamond: 57.58 % → 70.20 % (+12.62 pp) · LiveCodeBench v6 (Pass@1): 32.00 % → 39.43 % (+7.43 pp) · BFCL V4 Single-Turn AST: 82.33 % → 83.67 % (+1.34 pp, Non-Live: 88.44 %, Irrelevance: +33.39 pp) · Industrial Automation (PLC/CNC): +3.22 pp · 15-Benchmark Mean: 69.24 % → 70.26 % (+1.02 pp) |
| Finnish language quality | Native Bilingual (fi & en): Kotus-verified Finnish inflection, technical engineering prose, and zero-sycophancy epistemic discipline |
| Quantization & Footprint | NVIDIA Blackwell NVFP4 (compressed-tensors, 6 shards, 24.1 GB) — BF16 attention + shared w13 (gate_proj + up_proj) weight_global_scale across all 10,160 routed experts (0 scale mismatches) |
| VRAM Requirement | ~14.5 GB / GPU (TP=2 on 2x RTX 5090 32GB with 262,144 context & FP8 KV cache) |
| What was NOT changed | 27-layer multimodal vision encoder (model.visual.*, 333 tensors), Layer 40 MTP speculative head (mtp.*, 779 tensors), native 262,144 context window |
| Primary use case | Industrial automation (Siemens TIA Portal SCL, Beckhoff TwinCAT 3 ST, Fanuc/Sinumerik CNC G-code, OPC UA), software engineering & competitive coding, multi-language <tool_call> agents, PhD STEM, clinical medicine (MedQA 85.39 %), and autonomous dataset quality inspection/surgery |
| Tested environment | 2x NVIDIA GeForce RTX 5090 32GB Blackwell (sm_120, TP=2, CUDAGraphs enabled), vLLM, FlashInfer |