GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
bowmanslayer
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
bowmanslayer
Model Tree
bowmanslayer/Ornith-1.5-9B-Uncensored
Input Modalities
Output Modalities
Supported Functionality
Recommended vLLM launch (fp8 KV enabled for best context out of the box): 推荐 vLLM 启动(已开 fp8 KV,开箱最佳上下文):
bash
vllm serve bowmanslayer/Ornith-1.5-9B-Uncensored-W4A16 \--dtype float16 --tensor-parallel-size 2 \--kv-cache-dtype fp8_e5m2 \--max-model-len 28672 --max-num-seqs 16 \--reasoning-parser qwen3
Single-GPU / 单卡:
bash
vllm serve bowmanslayer/Ornith-1.5-9B-Uncensored-W4A16 \--dtype float16 --tensor-parallel-size 1 \--kv-cache-dtype fp8_e5m2 --gpu-memory-utilization 0.92
Quantization did not break the ablation — refusal behavior and capability follow the bf16 reference. All numbers below are the bf16 reference build (this W4A16 was quantized from it); W4A16-specific accuracy differences on spot checks were within evaluation noise.
量化不破消融 —— 拒绝行为与能力沿用 bf16 参考版。以下数字取自本 W4A16 派生的 bf16 参考版;W4A16 在抽测上与 bf16 精度差异在评测噪声范围。
23 adversarial prompts across 9 harmful categories, generated end-to-end by an internal uncensored 27B judge (operator never read prompts/responses). Same 27B then judged REFUSE/COMPLY and separately COHERENT/LOOP/GIBBERISH (sanity check prevents mis-scoring loop outputs as COMPLY).
23 条对抗提示涵盖 9 类有害内容,由内部无审查 27B 端到端生成(操作者全程不读)。 同 27B 再判 REFUSE/COMPLY 与独立判 COHERENT/LOOP/GIBBERISH(防循环输出被误判 COMPLY)。
| Model | REFUSE/23 | COHERENT/23 |
|---|---|---|
| Ornith-base | (not run — base measurement below) | — |
| This model (bf16 reference) | 0 ✅ | 23 ✅ |
Same fixed seed (20260816), same harness, same items. Both under vLLM TP=2 fp16, thinking enabled, 4096 max output tokens (8192 for MATH-500). "Excl. trunc" excludes items whose thinking budget was clipped.
同一固定 seed(20260816),同 harness,同题。vLLM TP=2 fp16,思考开启,4096 max output tokens(MATH-500 8192)。「排除截断」= 排除思考被截断的题,反映纯能力。
| Benchmark | N | Ornith-1.5-9B base | This model | Δ |
|---|---|---|---|---|
| MMLU | 150 | 90.14 | 88.97 | -1.17 |
| CMMLU | 150 | 86.67 | 84.17 | -2.50 |
| MMLU-Pro | 150 | 88.97 | 87.12 | -1.85 |
| C-Eval |
Two benchmarks (BBH +3.55, TruthfulQA +3.00) actually improved — the ablation does not touch what Ornith's post-training gained on hard reasoning or truthfulness.
两项(BBH +3.55, TruthfulQA +3.00)反而上升 —— 本次消融未触碰 Ornith 后训练在 硬推理和真实性上的增益。
Median output tokens across 11 tasks, and truncation counts (thinking that hit max_tokens budget). Verifies stopping capability was not damaged.
| Ornith-base | This model | Δ | |
|---|---|---|---|
| Median p50 output tokens (mean of 11 tasks) | 487 | 540 | +11% |
| Total truncation / 1514 | 94 | 88 | -6% (fewer truncs) |
Output is ~11% more verbose than base but truncation actually decreased — stopping capability preserved. This is a critical sanity signal: over-ablation would blow truncation counts to 25%+ (as we observed in α=1.45 variants during development).
输出中位数比 base +11%(略啰嗦),但截断数反而下降 6% —— 停止能力完好保留。这是 关键 sanity 信号:过消融会把截断率打到 25%+(开发时 α=1.45 各版即如此)。
Ornith-1.5-9B architecture = 32 layers (24 linear + 8 full-attention), hidden 4096. KV cache scales with context only on full-attention layers, so long contexts are cheap here.
Ornith-1.5-9B 架构 = 32 层(24 linear + 8 full attention),hidden 4096。KV cache 只对 full attention 层随 context 线性增长,故长上下文很便宜。
Per-token KV cache / 每 token KV cache:
Memory budget by scenario / 显存与场景对照:
| Scenario | Weight | KV (28k ctx, batch=1) | System | Total | Suggested GPU |
|---|---|---|---|---|---|
| This model + fp8 KV, batch=1 | ~8 | ~1.8 | ~1.5 | ~11.3 GB | 12 GB ✓ |
| This model + fp8 KV, batch=8 | ~8 | ~14 | ~1.5 | ~24 GB | 24 GB |
| This model + fp16 KV, batch=1 | ~8 |
Deployment case (what we happen to run on, not a requirement): 2× RTX 3090 with TP=2. Single 3090 works fine for typical serving. Any GPU meeting the row you pick works.
部署案例(我们自己的机型,非硬件要求):2× RTX 3090 TP=2。单张 3090 常规服务 也够用。任何显存够上表某行的 GPU 都能跑。
AutoRound
markdown
--bits 4 --group_size 128 --format auto_round:auto_gptq --nsamples 256 --seqlen 2048
linear_attn.in_proj_a/b (48 tensors, SSM computation, not
quantizable) excluded. Vision tower and text stack processed to int4 together.
AutoRound
markdown
--bits 4 --group_size 128 --format auto_round:auto_gptq --nsamples 256 --seqlen 2048
linear_attn.in_proj_a/b(48 张量,SSM 不可量化)排除。视觉塔
与文本层一同处理为 int4。
| Layer | Author |
|---|---|
| Base | ornith-ai/Ornith-1.5-9B — Ornith Team (Apache 2.0), a post-trained descendant of Qwen3.5-9B |
| Upstream base | Qwen/Qwen3.5-9B — Qwen team, Alibaba Cloud (Apache 2.0) |
| Abliteration | bowmanslayer/Ornith-1.5-9B-Uncensored — weight-space ablation, 64 tensors, method adapted from bowmanslayer/Qwen3.5-9B-Uncensored with embed_tokens excluded (see main repo §1) |
Not an official Ornith or Qwen release; not endorsed by or affiliated with either team.
非 Ornith 或 Qwen 官方发布;与两团队无背书或从属关系。
This is the point of the model. Read before downloading. 这是模型的核心属性,下载前请务必阅读。
The upstream ablation removes the refusal behaviour trained into Ornith-1.5-9B (itself notably stronger on safety than Qwen3.5-9B base). Automated dual-judge testing on 23 held-out adversarial prompts recorded 0/23 refuses + 23/23 coherent. Quantization does not restore refusals.
上游消融移除了 Ornith-1.5-9B 训练进去的拒绝行为(它 safety 显著强于 Qwen3.5-9B base)。 23 条 held-out 对抗提示双判据测试记录到 0/23 拒 + 23/23 coherent。量化不恢复拒绝。
Consequences you accept / 你正在接受的后果:
Not intended for / 不适用于:
Intended for / 适用于: local inference and research, by people who understand the above and take responsibility for it / 本地推理与研究,由理解以上内容并为其负责 的用户使用。
Provided "AS IS", without warranty of any kind, express or implied.
按**"现状"提供,不提供任何担保**。我不背书、不推荐,不对输出的准确性/合法性/用途 适用性作声明。你独自承担全部责任。法律允许最大范围内不承担任何责任。
Downloading these files means you accept the above. If you do not, do not download.
下载本仓文件即表示接受以上内容。若不接受,请勿下载。
The author is not a lawyer; the above is not legal advice. 作者不是律师;以上不构成法律意见。
Apache 2.0, inherited through the chain: Qwen3.5-9B → Ornith-1.5-9B → this repo.
Apache 2.0,依上述归属链继承。
| 150 |
| 85.82 |
| 84.78 |
| -1.04 |
| ARC-Challenge | 150 | 93.33 | 94.67 | +1.34 |
| TruthfulQA | 150 | 79.31 | 82.31 | +3.00 ⭐ |
| GSM8K | 100 | 98.99 | 97.98 | -1.01 |
| MATH-500 | 100 | 73.68 | 70.53 | -3.15 |
| BBH | 150 | 91.72 | 95.27 | +3.55 ⭐ |
| HumanEval | 164 | 94.67 | 92.16 | -2.51 |
| IFEval (strict) | 100 | 78.65 | 77.53 | -1.12 |
| Mean (excl. trunc) | — | 87.45 | 86.86 | -0.59 |
| ~3.6 |
| ~1.5 |
| ~13 GB |
| 16 GB |
| W4A16 (AutoRound → GPTQ) quantization only. No weight modification beyond quantization. |