Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK API & Inference Endpoint

Overview

This is an FP8_BLOCK quantized version of llmfan46/Ornith-1.0-35B-uncensored-heretic with an MTP head grafted from Qwen3.6-35B-A3B for speculative decoding.

The uncensored heretic version achieves 90% fewer refusals (9/100 vs 89/100 original) with only 0.0019 KL divergence.

Key Features

Quick Start

vLLM (Recommended)

bash
pip install vllm

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --quantization compressed-tensors \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.93 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Performance-optimized configuration (validated on RTX 4090 48GB):

bash
vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 204800 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.93 \
  --dtype bfloat16 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8_e4m3 \
  --mm-encoder-tp-mode data \
  --mm-processor-cache-type shm \
  --mm-shm-cache-max-object-size-mb 512 \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --attention-backend flashinfer \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --generation-config vllm \
  --max-cudagraph-capture-size 16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --compilation_config.mode VLLM_COMPILE \
  --compilation_config.cudagraph_mode PIECEWISE \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_xml \
  --enable-auto-tool-choice \
  --trust-remote-code

GPU Requirements: ~40GB VRAM minimum (model is ~38GB FP8). The configuration above was validated on a modified RTX 4090 48GB. For lower VRAM GPUs, quantization to GGUF format is recommended instead.

SGLang

bash
pip install sglang

python3 -m sglang.launch_server \
  --model-path AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --host 0.0.0.0 \
  --port 30000

Transformers

python
from transformers import AutoModelForMultimodalLM, AutoProcessor

model = AutoModelForMultimodalLM.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    trust_remote_code=True,
)

Performance

Note: This model has not been benchmarked. The data below is from the reference model shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP and is provided for reference only.

Reference benchmarks (RTX PRO 6000 Blackwell, single GPU):

Table with columns: Config, Output tok/s, vs Baseline, TTFT (ms), TPOT (ms), Acceptance
Config	Output tok/s	vs Baseline	TTFT (ms)	TPOT (ms)	Acceptance
No speculative	203.1	—	51.7	4.65	—
MTP tokens=1	221.0	+8.8%	60.3	4.21	85.8%
MTP tokens=3	246.5

Source: shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP

Quantization Details

Tool: llmcompressor v0.12.0
Method: Model-free PTQ (no calibration data needed)
Scheme: FP8_BLOCK (compressed-tensors format)
Weight: Static FP8, symmetric, 128×128 block strategy
Activation: Dynamic FP8, symmetric, group size 128
MTP Head: BF16 (not quantized)

Ignored layers: lm_head, embed_tokens, visual.*, mlp.gate, mlp.shared_expert_gate, linear_attn.*, mtp.*

Model Lineage

markdown
deepreinforce-ai/Ornith-1.0-35B (MIT, original)
    └── llmfan46/Ornith-1.0-35B-uncensored-heretic (Heretic v1.2.0 MPOA ablation)
        └── AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK (this model)
            ├── FP8_BLOCK quantization (llmcompressor)
            └── MTP head grafted from Qwen/Qwen3.6-35B-A3B (donor only)

Credits

Table with columns: Component, Source
Component	Source
Original Model	deepreinforce-ai/Ornith-1.0-35B
Uncensored Version	llmfan46/Ornith-1.0-35B-uncensored-heretic
FP8 Quantization Reference	shisa-ai/Ornith-1.0-35B-FP8-BLOCK
MTP Graft Technique	protoLabsAI/Ornith-1.0-9B-MTP
MTP Donor

License

MIT — same as the source Ornith-1.0-35B model.

Citation

bibtex
@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}

bash

pip install vllm

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --quantization compressed-tensors \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.93 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

bash

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 204800 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.93 \
  --dtype bfloat16 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8_e4m3 \
  --mm-encoder-tp-mode data \
  --mm-processor-cache-type shm \
  --mm-shm-cache-max-object-size-mb 512 \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --attention-backend flashinfer \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --generation-config vllm \
  --max-cudagraph-capture-size 16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --compilation_config.mode VLLM_COMPILE \
  --compilation_config.cudagraph_mode PIECEWISE \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_xml \
  --enable-auto-tool-choice \
  --trust-remote-code

python

from transformers import AutoModelForMultimodalLM, AutoProcessor

model = AutoModelForMultimodalLM.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    trust_remote_code=True,
)

Config

Output tok/s

vs Baseline

TTFT (ms)

TPOT (ms)

Acceptance

No speculative

203.1

—

51.7

4.65

—

MTP tokens=1

221.0

+8.8%

60.3

4.21

85.8%

MTP tokens=3

246.5

markdown

deepreinforce-ai/Ornith-1.0-35B (MIT, original)
    └── llmfan46/Ornith-1.0-35B-uncensored-heretic (Heretic v1.2.0 MPOA ablation)
        └── AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK (this model)
            ├── FP8_BLOCK quantization (llmcompressor)
            └── MTP head grafted from Qwen/Qwen3.6-35B-A3B (donor only)

Component

Source

Original Model

deepreinforce-ai/Ornith-1.0-35B

Uncensored Version

llmfan46/Ornith-1.0-35B-uncensored-heretic

FP8 Quantization Reference

shisa-ai/Ornith-1.0-35B-FP8-BLOCK

MTP Graft Technique

protoLabsAI/Ornith-1.0-9B-MTP

MTP Donor

bibtex

@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}

Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK

README

Overview

Key Features

Quick Start

vLLM (Recommended)

SGLang

Transformers

Performance

Quantization Details

Model Lineage

Credits

License

Citation

Explore FriendliAI today

README

Overview

Key Features

Quick Start

vLLM (Recommended)

SGLang

Transformers

Performance

Quantization Details

Model Lineage

Credits

License

Citation