MiMo-Ornith-9B-AGSI API & Inference Endpoint

📌 Executive Summary

MiMo-Ornith-9B-AGSI is a non-linear parameter-space synthesis of two leading fine-tuned models derived from the Qwen 9B architecture:

XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B: A distillation checkpoint optimized for dense mathematical deduction, SWE-bench verified programmatic problem-solving, and long-horizon chain-of-thought (CoT) reasoning.
ornith-ai/Ornith-1.5-9B: A reinforcement-learning-driven agentic model specializing in terminal/CLI mastery, autonomous bash execution, self-debugging loops, and GrandCode algorithmic generation.

Rather than relying on naive linear averaging (LERP) or global spherical interpolation (SLERP)—which flatten parameter tensors into uncalibrated vectors—this model was fused using Adaptive Geodesic Spectral Interpolation (AGSI). AGSI executes row-wise hyperspherical geodesics on decoupled directional manifolds, enforces second-order spectral energy conservation, shields against anti-phase gradient interference, and dynamically modulates parameters across depth via a $C 2$ -continuous quintic smoothstep curve.

🔬 The Mathematics of AGSI

Standard model interpolation techniques often suffer from:

Frobenius Attenuation: Convex parameter averaging causes systematic shrinkage of matrix norms ( $∥ (1 - t) W A + t W B ∥ F < ∥$ ), resulting in signal degradation across 32 transformer layers.

AGSI resolves these issues through a five-stage manifold interpolation framework:

AGSI pipeline

1. Direction-Magnitude (DoRA) Decoupling

Linear layers compute transformations $y = x W T$ , where individual rows $W i \in R D in $ represent the hyperplanes of specific neurons. AGSI isolates radial feature scale from directional orientation:

$m (i) = ∥ W (i) ∥ 2 = \sum j = 1$

$u (i) = m (i) + ϵ W (i) , where u$

By constraining directional updates to the unit hypersphere $S D in - 1$ , the model preserves the angular separation of neuron receptive fields.

2. Row-Wise Hyperspherical Geodesics ( $S D - 1$ )

For each individual neuron row $i$ , the angular geodesic distance between checkpoint trajectories is calculated directly in its tangent space:

$θ i = arccos (clamp (⟨ u A (i) , u B (i) $

Rather than using Euclidean displacement, the directional basis moves along the great-circle arc:

$u fused (i) = c A (i) u A (i) +$

where the geodesic velocity coefficients are defined as:

$c A (i) = s i n θ i + ϵ s i n ((1 - t) θ$

If $sin θ i \to 0$ (quasi-collinear vectors), the interpolation smoothly transitions to normalized linear blending:

$lim θ \to 0 u fused (i) = (1 - t) u A (i) $

3. Anti-Phase Interference Shielding

When RL optimization (Ornith) and distillation gradients (MiMo) pull in opposing directions ( $θ i > 9 0 \circ, cos θ i < 0$ ), standard spherical combination results in substantial cancellation:

$∥ u A + u B ∥ = 2 + 2 cos θ <$

AGSI monitors the directional dot product against a critical interference bound ( $τ = - 0.05$ , corresponding to $θ \approx 92.8 6 \circ$ ). If destructive cancellation occurs:

$Conflict Condition: ⟨ u A (i) , u B (i) ⟩ < τ$

$γ A (i) = m A (i) + m B (i$

0.85\, u_A^{(i)} + 0.15\, u_B^{(i)}, & \text{if } \gamma_A^{(i)} \ge 0.5 \\ 0.15\, u_A^{(i)} + 0.85\, u_B^{(i)}, & \text{if } \gamma_A^{(i)} < 0.5 \end{cases}$$ This dominance gate projects conflicting features toward the checkpoint exhibiting higher parameter variance, preventing the formation of dormant neurons. ![Geodesic SLERP vs anti-phase dominance gate](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/dark/vectors.png#hf-dark-mode-only) ![Geodesic SLERP vs anti-phase dominance gate](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/light/vectors.png#hf-light-mode-only) --- ### 4. Quadratic Spectral Energy Invariant In deep autoregressive networks normalized by RMSNorm, layer output variance relies heavily on parameter norm preservation: $$\mathbb{E}[\|W x\|_2^2] \approx \frac{1}{D_{\text{in}}} \|W\|_F^2 \, \mathbb{E}[\|x\|_2^2]$$ To maintain stable forward-pass activations without logit saturation or signal decay, AGSI scales the interpolated direction vector by the root-mean-square energy: $$m_{\text{target}}^{(i)} = \sqrt{(1 - t)(m_A^{(i)})^2 + t(m_B^{(i)})^2}$$ $$W_{\text{candidate}}^{(i)} = m_{\text{target}}^{(i)} \cdot \frac{u_{\text{fused}}^{(i)}}{\|u_{\text{fused}}^{(i)}\|_2}$$ Finally, a global Frobenius norm correction is applied across the full matrix: $$W_{\text{final}} = W_{\text{candidate}} \times \left( \frac{\sqrt{(1 - t)\|W_A\|_F^2 + t\|W_B\|_F^2}}{\|W_{\text{candidate}}\|_F + \epsilon} \right)$$ --- ### 5. Quintic Smoothstep Depth & Functional Block Routing The mixing coefficient $t$ is non-static. It is governed by a $C^2$-continuous quintic polynomial function across the 32 transformer layers, augmented by functional block biases: $$\xi = \frac{l}{L - 1} \in [0, 1], \quad l \in \{0, 1, \dots, 31\}$$ $$S(\xi) = 6\xi^5 - 15\xi^4 + 10\xi^3$$ $$t(l, \text{block}) = \text{clamp}\left(t_{\text{base}} + \Delta t \cdot \left(S(\xi) - 0.5\right) + \beta_{\text{block}}, \, 0.0, \, 1.0\right)$$ ![Mixing ratio across transformer depth](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/dark/chart.png#hf-dark-mode-only) ![Mixing ratio across transformer depth](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/light/chart.png#hf-light-mode-only) #### Layer Profile Breakdown: * **Layers 0–7 ($t \approx 0.41 - 0.44$) — Syntactic Anchoring**: Biased toward MiMo-V2.6 to preserve input parsing, stable token embeddings, and baseline representations. * **Layers 8–23 ($t \approx 0.45 - 0.51$) — Cognitive & Algorithmic Core**: Near-equilibrium blending. Mathematical deductions and algorithm planning synthesize with Ornith's tool-use reasoning. * **Layers 24–31 ($t \approx 0.52 - 0.55$) — Action Policy & Output Heads**: Biased toward Ornith-1.5 to prioritize terminal command generation, tool-calling syntax, and execution policies. * **Attention vs. MLP Decoupling**: * **Attention Projections (`q, k, v, o, linear_attn`)**: $\beta_{\text{attn}} = +0.04$ (Ornith priority for context selection and environment state tracking). * **MLP / Feed-Forward (`gate, up, down_proj`)**: $\beta_{\text{mlp}} = -0.04$ (MiMo priority for factual retention and associative coding memories). --- ## ⚙️ AGSI Configuration Profile ```python # The Golden SOTA Preset applied during parameter synthesis BASE_ORNITH_RATIO = 0.48 # Foundational balance (52% MiMo / 48% Ornith) DEPTH_MODULATION = 0.14 # Dynamic span across transformer depth ATTN_ROUTING_BIAS = 0.04 # Attention block offset (favors Ornith) MLP_ROUTING_BIAS = -0.04 # MLP block offset (favors MiMo) ANTI_PHASE_THRESHOLD = -0.05 # Critical angle cutoff (θ = 92.86°) SPECTRAL_ENERGY_EXPONENT = 2.0 # Second-order L2 moment conservation ``` --- ## 🚀 Deployment & Inference This model is compatible with systems supporting the Qwen 9B architecture (e.g., standard Hugging Face `transformers`, `vLLM`, `SGLang`). ### Serving with vLLM (Recommended) To run the model with vLLM, including support for streaming reasoning tokens (`<think>`) and agentic tool calls: ```bash vllm serve DEST_REPO_ID \ --port 8000 \ --model DEST_REPO_ID \ --trust-remote-code \ --tensor-parallel-size 1 \ --max-model-len 131072 \ --gpu-memory-utilization 0.92 \ --reasoning-parser qwen3 \ --tool-call-parser hermes ``` ### Python Inference via Transformers ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "DEST_REPO_ID" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True ) messages = [ { "role": "system", "content": "You are a master systems engineer and competitive programmer. Solve problems using precise step-by-step reasoning enclosed in <think> tags, then execute terminal commands or output production code." }, { "role": "user", "content": "Write an optimized eBPF program in C that traces network socket latency outliers (>100ms) and provides a Python BCC analysis script." } ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=4096, temperature=0.6, top_p=0.95, repetition_penalty=1.05 ) response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True) print(response) ``` --- ## 🎯 Prompt Formatting & Chat Template This model uses the unified Qwen chat template, configured to separate internal reasoning steps from final outputs: ```xml <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>user Write a bash script to monitor memory usage.<|im_end|> <|im_start|>assistant <think> 1. Identify target metrics: available vs. used memory. 2. Use /proc/meminfo or 'free -m' for portability. 3. Handle logging and alert thresholds. </think> Here is the monitoring script: ``` ```bash #!/usr/bin/env bash set -euo pipefail THRESHOLD=85 CURRENT=$(free | awk '/Mem:/ {printf("%.0f"), $3/$2 * 100}') if [ "$CURRENT" -gt "$THRESHOLD" ]; then echo "WARNING: Memory usage at ${CURRENT}%" fi ``` `<|im_end|>` --- ## ⚖️ License & Attribution * **Base Model Checkpoints**: * Checkpoint A: [XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B) * Checkpoint B: [ornith-ai/Ornith-1.5-9B](https://huggingface.co/ornith-ai/Ornith-1.5-9B) * **Underlying Architecture**: Qwen Series (Alibaba Cloud) * **License**: Apache 2.0

📌 Executive Summary

MiMo-Ornith-9B-AGSI is a non-linear parameter-space synthesis of two leading fine-tuned models derived from the Qwen 9B architecture:

XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B: A distillation checkpoint optimized for dense mathematical deduction, SWE-bench verified programmatic problem-solving, and long-horizon chain-of-thought (CoT) reasoning.
ornith-ai/Ornith-1.5-9B: A reinforcement-learning-driven agentic model specializing in terminal/CLI mastery, autonomous bash execution, self-debugging loops, and GrandCode algorithmic generation.

🔬 The Mathematics of AGSI

Standard model interpolation techniques often suffer from:

Frobenius Attenuation: Convex parameter averaging causes systematic shrinkage of matrix norms ( $∥ (1 - t) W A + t W B ∥ F < ∥$ ), resulting in signal degradation across 32 transformer layers.

AGSI resolves these issues through a five-stage manifold interpolation framework:

AGSI pipeline

1. Direction-Magnitude (DoRA) Decoupling

$m (i) = ∥ W (i) ∥ 2 = \sum j = 1$

$u (i) = m (i) + ϵ W (i) , where u$

By constraining directional updates to the unit hypersphere $S D in - 1$ , the model preserves the angular separation of neuron receptive fields.

2. Row-Wise Hyperspherical Geodesics ( $S D - 1$ )

For each individual neuron row $i$ , the angular geodesic distance between checkpoint trajectories is calculated directly in its tangent space:

$θ i = arccos (clamp (⟨ u A (i) , u B (i) $

Rather than using Euclidean displacement, the directional basis moves along the great-circle arc:

$u fused (i) = c A (i) u A (i) +$

where the geodesic velocity coefficients are defined as:

$c A (i) = s i n θ i + ϵ s i n ((1 - t) θ$

If $sin θ i \to 0$ (quasi-collinear vectors), the interpolation smoothly transitions to normalized linear blending:

$lim θ \to 0 u fused (i) = (1 - t) u A (i) $

3. Anti-Phase Interference Shielding

When RL optimization (Ornith) and distillation gradients (MiMo) pull in opposing directions ( $θ i > 9 0 \circ, cos θ i < 0$ ), standard spherical combination results in substantial cancellation:

$∥ u A + u B ∥ = 2 + 2 cos θ <$

AGSI monitors the directional dot product against a critical interference bound ( $τ = - 0.05$ , corresponding to $θ \approx 92.8 6 \circ$ ). If destructive cancellation occurs:

$Conflict Condition: ⟨ u A (i) , u B (i) ⟩ < τ$

$γ A (i) = m A (i) + m B (i$

0.85\, u_A^{(i)} + 0.15\, u_B^{(i)}, & \text{if } \gamma_A^{(i)} \ge 0.5 \\ 0.15\, u_A^{(i)} + 0.85\, u_B^{(i)}, & \text{if } \gamma_A^{(i)} < 0.5 \end{cases}$$ This dominance gate projects conflicting features toward the checkpoint exhibiting higher parameter variance, preventing the formation of dormant neurons. ![Geodesic SLERP vs anti-phase dominance gate](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/dark/vectors.png#hf-dark-mode-only) ![Geodesic SLERP vs anti-phase dominance gate](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/light/vectors.png#hf-light-mode-only) --- ### 4. Quadratic Spectral Energy Invariant In deep autoregressive networks normalized by RMSNorm, layer output variance relies heavily on parameter norm preservation: $$\mathbb{E}[\|W x\|_2^2] \approx \frac{1}{D_{\text{in}}} \|W\|_F^2 \, \mathbb{E}[\|x\|_2^2]$$ To maintain stable forward-pass activations without logit saturation or signal decay, AGSI scales the interpolated direction vector by the root-mean-square energy: $$m_{\text{target}}^{(i)} = \sqrt{(1 - t)(m_A^{(i)})^2 + t(m_B^{(i)})^2}$$ $$W_{\text{candidate}}^{(i)} = m_{\text{target}}^{(i)} \cdot \frac{u_{\text{fused}}^{(i)}}{\|u_{\text{fused}}^{(i)}\|_2}$$ Finally, a global Frobenius norm correction is applied across the full matrix: $$W_{\text{final}} = W_{\text{candidate}} \times \left( \frac{\sqrt{(1 - t)\|W_A\|_F^2 + t\|W_B\|_F^2}}{\|W_{\text{candidate}}\|_F + \epsilon} \right)$$ --- ### 5. Quintic Smoothstep Depth & Functional Block Routing The mixing coefficient $t$ is non-static. It is governed by a $C^2$-continuous quintic polynomial function across the 32 transformer layers, augmented by functional block biases: $$\xi = \frac{l}{L - 1} \in [0, 1], \quad l \in \{0, 1, \dots, 31\}$$ $$S(\xi) = 6\xi^5 - 15\xi^4 + 10\xi^3$$ $$t(l, \text{block}) = \text{clamp}\left(t_{\text{base}} + \Delta t \cdot \left(S(\xi) - 0.5\right) + \beta_{\text{block}}, \, 0.0, \, 1.0\right)$$ ![Mixing ratio across transformer depth](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/dark/chart.png#hf-dark-mode-only) ![Mixing ratio across transformer depth](https://huggingface.co/OliviaRossi/MiMo-Ornith-9B-AGSI/resolve/main/assets/light/chart.png#hf-light-mode-only) #### Layer Profile Breakdown: * **Layers 0–7 ($t \approx 0.41 - 0.44$) — Syntactic Anchoring**: Biased toward MiMo-V2.6 to preserve input parsing, stable token embeddings, and baseline representations. * **Layers 8–23 ($t \approx 0.45 - 0.51$) — Cognitive & Algorithmic Core**: Near-equilibrium blending. Mathematical deductions and algorithm planning synthesize with Ornith's tool-use reasoning. * **Layers 24–31 ($t \approx 0.52 - 0.55$) — Action Policy & Output Heads**: Biased toward Ornith-1.5 to prioritize terminal command generation, tool-calling syntax, and execution policies. * **Attention vs. MLP Decoupling**: * **Attention Projections (`q, k, v, o, linear_attn`)**: $\beta_{\text{attn}} = +0.04$ (Ornith priority for context selection and environment state tracking). * **MLP / Feed-Forward (`gate, up, down_proj`)**: $\beta_{\text{mlp}} = -0.04$ (MiMo priority for factual retention and associative coding memories). --- ## ⚙️ AGSI Configuration Profile ```python # The Golden SOTA Preset applied during parameter synthesis BASE_ORNITH_RATIO = 0.48 # Foundational balance (52% MiMo / 48% Ornith) DEPTH_MODULATION = 0.14 # Dynamic span across transformer depth ATTN_ROUTING_BIAS = 0.04 # Attention block offset (favors Ornith) MLP_ROUTING_BIAS = -0.04 # MLP block offset (favors MiMo) ANTI_PHASE_THRESHOLD = -0.05 # Critical angle cutoff (θ = 92.86°) SPECTRAL_ENERGY_EXPONENT = 2.0 # Second-order L2 moment conservation ``` --- ## 🚀 Deployment & Inference This model is compatible with systems supporting the Qwen 9B architecture (e.g., standard Hugging Face `transformers`, `vLLM`, `SGLang`). ### Serving with vLLM (Recommended) To run the model with vLLM, including support for streaming reasoning tokens (`<think>`) and agentic tool calls: ```bash vllm serve DEST_REPO_ID \ --port 8000 \ --model DEST_REPO_ID \ --trust-remote-code \ --tensor-parallel-size 1 \ --max-model-len 131072 \ --gpu-memory-utilization 0.92 \ --reasoning-parser qwen3 \ --tool-call-parser hermes ``` ### Python Inference via Transformers ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "DEST_REPO_ID" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True ) messages = [ { "role": "system", "content": "You are a master systems engineer and competitive programmer. Solve problems using precise step-by-step reasoning enclosed in <think> tags, then execute terminal commands or output production code." }, { "role": "user", "content": "Write an optimized eBPF program in C that traces network socket latency outliers (>100ms) and provides a Python BCC analysis script." } ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=4096, temperature=0.6, top_p=0.95, repetition_penalty=1.05 ) response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True) print(response) ``` --- ## 🎯 Prompt Formatting & Chat Template This model uses the unified Qwen chat template, configured to separate internal reasoning steps from final outputs: ```xml <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>user Write a bash script to monitor memory usage.<|im_end|> <|im_start|>assistant <think> 1. Identify target metrics: available vs. used memory. 2. Use /proc/meminfo or 'free -m' for portability. 3. Handle logging and alert thresholds. </think> Here is the monitoring script: ``` ```bash #!/usr/bin/env bash set -euo pipefail THRESHOLD=85 CURRENT=$(free | awk '/Mem:/ {printf("%.0f"), $3/$2 * 100}') if [ "$CURRENT" -gt "$THRESHOLD" ]; then echo "WARNING: Memory usage at ${CURRENT}%" fi ``` `<|im_end|>` --- ## ⚖️ License & Attribution * **Base Model Checkpoints**: * Checkpoint A: [XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B) * Checkpoint B: [ornith-ai/Ornith-1.5-9B](https://huggingface.co/ornith-ai/Ornith-1.5-9B) * **Underlying Architecture**: Qwen Series (Alibaba Cloud) * **License**: Apache 2.0

MiMo-Ornith-9B-AGSI

README

📌 Executive Summary

🔬 The Mathematics of AGSI

1. Direction-Magnitude (DoRA) Decoupling

2. Row-Wise Hyperspherical Geodesics ( $S D - 1$ )

3. Anti-Phase Interference Shielding

Explore FriendliAI today

README

📌 Executive Summary

🔬 The Mathematics of AGSI

1. Direction-Magnitude (DoRA) Decoupling

2. Row-Wise Hyperspherical Geodesics ( $S D - 1$ )

3. Anti-Phase Interference Shielding

MiMo-Ornith-9B-AGSI

README

📌 Executive Summary

🔬 The Mathematics of AGSI

1. Direction-Magnitude (DoRA) Decoupling

2. Row-Wise Hyperspherical Geodesics (SD−1)

3. Anti-Phase Interference Shielding

Explore FriendliAI today

README

📌 Executive Summary

🔬 The Mathematics of AGSI

1. Direction-Magnitude (DoRA) Decoupling

2. Row-Wise Hyperspherical Geodesics (SD−1)

3. Anti-Phase Interference Shielding

2. Row-Wise Hyperspherical Geodesics ( $S D - 1$ )

2. Row-Wise Hyperspherical Geodesics ( $S D - 1$ )