Available Quantized Variants
All three repositories share the exact same SignRound-optimized weights (W4A16, group_size=32, symmetric) exported simultaneously into three runtime serialization schemas:
Quantization Methodology & Precision Architecture
Optimization Configuration
- Algorithm: Intel AutoRound (
SignRoundConfig)
- Precision & Granularity:
W4A16, group_size=32, sym=True
- Tuning Hyperparameters:
1200 iterations per block, batch_size=2, gradient_accumulate_steps=4 (effective global batch size = 8), seqlen=4096, nsamples=512
Mixed-Precision Layer Preservation (Hybrid Qwen3.5 Architecture)
Swift-1.5-Ornith-9B uses a hybrid Qwen3_5ForConditionalGeneration architecture combining 24 GatedDeltaNet linear-attention blocks with 8 full-attention blocks, a vision encoder, and a 1-layer Multi-Token Prediction (MTP) head. To prevent accuracy degradation in recurrent state updates, vision embeddings, and speculative drafting, 128/359 linear layers are quantized to INT4 while 231 sensitive modules remain in BF16:
Table with columns: Component, Precision, Details| Component | Precision | Details |
|---|
| Full-Attention & MLP Projections | INT4 (W4A16, g=32) | Quantized across language model blocks (q/k/v/o_proj, gate/up/down_proj) |
GatedDeltaNet (linear_attn) | BF16 | in_proj_a, in_proj_b, in_proj_qkv, in_proj_z, out_proj across all 24 linear-attention layers |
Vision Encoder (visual & merger) | BF16 | All 27 vision transformer blocks and patch merger preserved (quant_nontext_module=False) |
MTP Speculative Head (mtp) | BF16 | All 15 MTP tensors preserved in model_extra_tensors.safetensors for native speculative decoding |
Language Modeling Head (lm_head) | BF16 | Kept unquantized for logit precision |
Calibration Dataset Mix (512 Samples @ 4096 Tokens)
All 512 calibration sequences were verified to exceed 4,096 tokens (min=4,103, max=51,063) with chat templates and <think> traces intact:
Table with columns: Dataset, Samples, Share, Domain & Formatting| Dataset | Samples | Share | Domain & Formatting |
|---|
open-thoughts/OpenThoughts3-1.2M | 180 | 35.16% | STEM, math, and code reasoning traces (<think>) |
nvidia/OpenCodeReasoning (split_0) | 110 | 21.48% | Competitive coding with chain-of-thought reasoning |
openbmb/UltraData-SFT-Agent-2609 | 102 | 19.92% | Multi-turn agentic trajectories & XML tool definitions |
nvidia/OpenCodeInstruct | 70 | 13.67% | Direct code generation and execution instruction pairs |
HuggingFaceH4/ultrachat_200k | 50 | 9.77% | General multi-turn conversational alignment |
Serving with vLLM
Requires vLLM ≥ 0.19.1 and Transformers ≥ 5.8.1.
1. Standard High-Throughput Serving (200K Context + FP8 KV Cache)
Note: Replace Vishva007/Swift-1.5-Ornith-9B-W4A16-GPTQ below with -AutoRound or -LLM-Compressor depending on your preferred checkpoint format.
vllm serve Vishva007/Swift-1.5-Ornith-9B-W4A16-GPTQ \ --host 0.0.0.0 \ --port 8000 \ --trust-remote-code \ --max-model-len 200k \ --gpu-memory-utilization 0.80 \ --kv-cache-dtype fp8 \ --enable-prefix-caching \ --prefix-caching-hash-algo sha256 \ --enable-chunked-prefill \ --max-num-batched-tokens 8k \ --max-num-scheduled-tokens 8k \ --max-num-seqs 16 \ --max-num-active-seqs 8 \ --long-prefill-token-threshold 8384 \ --scheduler-reserve-full-isl \ --linear-backend auto \ --enable-flashinfer-autotune \ --async-scheduling \ --max-cudagraph-capture-size 256 \ --kv-offloading-size 8 \ --kv-offloading-backend native \ --reasoning-parser qwen3 \ --default-chat-template-kwargs '{"enable_thinking": true}' \ --enable-auto-tool-choice \ --tool-call-parser qwen3_xml \ --kv-transfer-config '{"kv_connector":"SimpleCPUOffloadConnector","kv_role":"kv_both","kv_connector_extra_config":{"cpu_bytes_to_use_per_rank":25769803776,"lazy_offload":false}}'
- Thinking Mode Toggle: The command above sets
'{"enable_thinking": true}' by default. Remove --default-chat-template-kwargs (or pass {"enable_thinking": false} per request via extra_body={"chat_template_kwargs": {"enable_thinking": false}}) to disable <think>...</think> reasoning traces routed to reasoning_content.
2. Enabling Speculative Decoding (MTP)
Because all 15 BF16 MTP head tensors are preserved in the checkpoint, you can append the MTP speculative decoding flag to any of the variants:
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
Recommended Sampling Parameters
- General & Reasoning Tasks:
temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
- Precise Coding & Agentic Tool Use:
temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
Python Client Example (OpenAI-Compatible API)
from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") response = client.chat.completions.create( model="Vishva007/Swift-1.5-Ornith-9B-W4A16-GPTQ", messages=[ {"role": "user", "content": "Write a Python function to verify if a directed graph is a DAG."} ], temperature=0.6, top_p=0.95, max_tokens=2048, extra_body={"chat_template_kwargs": {"enable_thinking": True}},) msg = response.choices[0].messageprint("Reasoning:", getattr(msg, "reasoning_content", None))print("Answer:", msg.content)
🚀 Deploy on RunPod
One-click launch environments pre-configured with PyTorch, CUDA, and dependencies for fine-tuning or quantization.
🎁 Need GPU compute? Sign up via RunPod and get 5–500 in free credits when you add your first $10.
PyTorch 2.15
Table with columns: Template, CUDA Version, Docker Image, Template ID, Deploy| Template | CUDA Version | Docker Image | Template ID | Deploy |
|---|
| PyTorch 2.15 (CUDA 13.2) | 13.2 | vishva123/cuda-13.2-pytorch-2.15-runpod | 0rsw686010 |  |
| PyTorch 2.15 (CUDA 13.4) | 13.4 | vishva123/cuda-13.4-pytorch-2.15-runpod | 0s0rhdfaqy |  |
PyTorch 2.14
Table with columns: Template, CUDA Version, Docker Image, Template ID, Deploy| Template | CUDA Version | Docker Image | Template ID | Deploy |
|---|
| PyTorch 2.14 (CUDA 12.6) | 12.6 | vishva123/cuda-12.6-pytorch-2.14-runpod | d7lxsa4w9m |  |
| PyTorch 2.14 (CUDA 13.0) | 13.0 | vishva123/cuda-13.0-pytorch-2.14-runpod | yk0y6j6rpg |  |
| PyTorch 2.14 (CUDA 13.2) | 13.2 | vishva123/cuda-13.2-pytorch-2.14-runpod | gsp4gwx0nw |  |
PyTorch 2.13
Table with columns: Template, CUDA Version, Docker Image, Template ID, Deploy| Template | CUDA Version | Docker Image | Template ID | Deploy |
|---|
| PyTorch 2.13 (CUDA 12.6) | 12.6 | vishva123/cuda-12.6-pytorch-2.13-runpod | gmlupxnxfk |  |
| PyTorch 2.13 (CUDA 13.0) | 13.0 | vishva123/cuda-13.0-pytorch-2.13-runpod | y3j8xvk4f4 |  |
| PyTorch 2.13 (CUDA 13.2) | 13.2 | vishva123/cuda-13.2-pytorch-2.13-runpod | vigpissn5w |  |
PyTorch 2.12
Table with columns: Template, CUDA Version, Docker Image, Template ID, Deploy| Template | CUDA Version | Docker Image | Template ID | Deploy |
|---|
| PyTorch 2.12 (CUDA 12.6) | 12.6 | vishva123/cuda-12.6-pytorch-2.12-runpod | ctmz86zmf0 |  |
| PyTorch 2.12 (CUDA 13.0) | 13.0 | vishva123/cuda-13.0-pytorch-2.12-runpod | qjko5yiwzi |  |
| PyTorch 2.12 (CUDA 13.2) | 13.2 | vishva123/cuda-13.2-pytorch-2.12-runpod | ifg6xmye0f |  |
License & Acknowledgements
@misc{swift-1.5-ornith-9b, title = {Swift 1.5 Ornith 9B}, author = {UkisAI}, year = {2026}, url = {[https://huggingface.co/ukisai/Swift-1.5-Ornith-9B](https://huggingface.co/ukisai/Swift-1.5-Ornith-9B)}} @misc{ornith_1_5, title = {{Ornith-1.5}: From Self-Scaffolding to Self-Improvement}, url = {[https://ornith.ai/ornith_1_5.html](https://ornith.ai/ornith_1_5.html)}, author = {{Ornith Team}}, year = {2026}}