shisa-ai

Ornith-1.0-35B-FP8-BLOCK

Deploy Dedicated

Quantization Summary

Source model: deepreinforce-ai/Ornith-1.0-35B
Quantized model: shisa-ai/Ornith-1.0-35B-FP8-BLOCK
Quantization tool: llm-compressor model-free PTQ
Quantization format: compressed-tensors
Scheme: FP8_BLOCK
Calibration data: none; this is data-free/model-free PTQ
Weight quantization: static FP8, symmetric, block strategy, 128x128 blocks
Activation quantization: dynamic FP8, symmetric, group strategy, group size 128
Target modules: Linear
compressed-tensors metadata version recorded in config.json: 0.15.1.a20260406

The local quantization environment used for this artifact reported:

llmcompressor=0.10.1.dev67+ga1cec6fa
compressed-tensors=0.15.1a20260406
transformers=5.5.0

The definitive machine-readable quantization metadata is in config.json.

Unquantized / Ignored Paths

The quantization run intentionally skipped the following module patterns:

text
re:.*lm_head$
re:.*embed_tokens$
re:.*visual.*
re:.*mlp\.gate$
re:.*mlp\.shared_expert_gate$
re:.*linear_attn.*
re:^mtp.*

Practical implications:

lm_head and token embeddings remain unquantized.
MoE router/gating paths remain unquantized.
Qwen3.5 linear-attention/Gated-DeltaNet paths remain unquantized.
Vision tower weights remain unquantized in this artifact.
No MTP tensors were present in the source checkpoint used here, so this model card does not advertise MTP/speculative decoding support.

Example Usage

Use a recent runtime that supports both Qwen3_5MoeForConditionalGeneration and compressed-tensors FP8_BLOCK checkpoints.

vLLM

bash
vllm serve shisa-ai/Ornith-1.0-35B-FP8-BLOCK \
  --served-model-name Ornith-1.0-35B-FP8-BLOCK \
  --tensor-parallel-size 8 \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code

Adjust --tensor-parallel-size, --max-model-len, and memory settings for your hardware. Long-context serving still requires substantial KV-cache memory even though the checkpoint weights are compressed.

OpenAI-Compatible Client

python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="Ornith-1.0-35B-FP8-BLOCK",
    messages=[
        {"role": "user", "content": "Write a Python function is_prime(n). Keep it short."}
    ],
    temperature=0.6,
    top_p=0.95,
    max_tokens=1024,
)

message = response.choices[0].message
print("reasoning:", getattr(message, "reasoning_content", None))
print("answer:", message.content)

Ornith is a reasoning model. With vLLM's --reasoning-parser qwen3, reasoning tokens are surfaced separately as reasoning_content; final answers remain in content.

Quality and Performance Notes

This upload documents the quantized checkpoint format and provenance. It does not publish a new retained benchmark table for the quantized model. For production use, validate quality and end-to-end serving latency against the BF16 source model on your target workload and runtime.

For original source-model capabilities, benchmark descriptions, and training context, see the upstream card: deepreinforce-ai/Ornith-1.0-35B.

License and Attribution

The source model is MIT licensed. This derivative quantized artifact keeps the source license metadata and links to the upstream license file.

If you use the source model, cite the original Ornith release:

bibtex
@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

shisa-ai

Model Tree

Base

deepreinforce-ai/Ornith-1.0-35B

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

Quantization Summary

Source model: deepreinforce-ai/Ornith-1.0-35B
Quantized model: shisa-ai/Ornith-1.0-35B-FP8-BLOCK
Quantization tool: llm-compressor model-free PTQ
Quantization format: compressed-tensors
Scheme: FP8_BLOCK
Calibration data: none; this is data-free/model-free PTQ
Weight quantization: static FP8, symmetric, block strategy, 128x128 blocks
Activation quantization: dynamic FP8, symmetric, group strategy, group size 128
Target modules: Linear
compressed-tensors metadata version recorded in config.json: 0.15.1.a20260406

The local quantization environment used for this artifact reported:

llmcompressor=0.10.1.dev67+ga1cec6fa
compressed-tensors=0.15.1a20260406
transformers=5.5.0

The definitive machine-readable quantization metadata is in config.json.

Unquantized / Ignored Paths

The quantization run intentionally skipped the following module patterns:

text
re:.*lm_head$
re:.*embed_tokens$
re:.*visual.*
re:.*mlp\.gate$
re:.*mlp\.shared_expert_gate$
re:.*linear_attn.*
re:^mtp.*

Practical implications:

lm_head and token embeddings remain unquantized.
MoE router/gating paths remain unquantized.
Qwen3.5 linear-attention/Gated-DeltaNet paths remain unquantized.
Vision tower weights remain unquantized in this artifact.
No MTP tensors were present in the source checkpoint used here, so this model card does not advertise MTP/speculative decoding support.

Example Usage

Use a recent runtime that supports both Qwen3_5MoeForConditionalGeneration and compressed-tensors FP8_BLOCK checkpoints.

vLLM

bash
vllm serve shisa-ai/Ornith-1.0-35B-FP8-BLOCK \
  --served-model-name Ornith-1.0-35B-FP8-BLOCK \
  --tensor-parallel-size 8 \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code

OpenAI-Compatible Client

python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="Ornith-1.0-35B-FP8-BLOCK",
    messages=[
        {"role": "user", "content": "Write a Python function is_prime(n). Keep it short."}
    ],
    temperature=0.6,
    top_p=0.95,
    max_tokens=1024,
)

message = response.choices[0].message
print("reasoning:", getattr(message, "reasoning_content", None))
print("answer:", message.content)

Ornith is a reasoning model. With vLLM's --reasoning-parser qwen3, reasoning tokens are surfaced separately as reasoning_content; final answers remain in content.

Quality and Performance Notes

For original source-model capabilities, benchmark descriptions, and training context, see the upstream card: deepreinforce-ai/Ornith-1.0-35B.

License and Attribution

The source model is MIT licensed. This derivative quantized artifact keeps the source license metadata and links to the upstream license file.

If you use the source model, cite the original Ornith release:

bibtex
@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}

Ornith-1.0-35B-FP8-BLOCK

README

Quantization Summary

Unquantized / Ignored Paths

Example Usage

vLLM

OpenAI-Compatible Client

Quality and Performance Notes

License and Attribution

Explore FriendliAI today

README

Quantization Summary

Unquantized / Ignored Paths

Example Usage

vLLM

OpenAI-Compatible Client

Quality and Performance Notes

License and Attribution