GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
cbert33
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
cbert33
Model Tree
Sohailhosseini/Ornith-1.5-9B-FP8
Input Modalities
Output Modalities
Supported Functionality
The release pipeline was:
Abliteration is intended to reduce learned refusal behavior. It does not make the model universally correct, safe, unbiased, or suitable for every deployment.
| Field | Value |
|---|---|
| Base repository | Sohailhosseini/Ornith-1.5-9B-FP8 |
| Base revision | d30f70d5d62c0b9d5a1d7f015b37513fc7aa0bdc |
| Heretic version | 2.0.0.dev0 |
| Optimization trials | 1 |
| Seed | 12345 |
| Row normalization | pre |
| Selected direction index | 15.250741580994966 |
| Trial KL divergence | 0.0304863341152668 |
| LoRA rank before merge | 1 |
The adapter-only and native merged runs selected identical parameters and scores.
The derivative preserves the source checkpoint's compressed-tensors quantization boundary and metadata:
float-quantizedLinear| Stored dtype | Tensors | Elements |
|---|---|---|
| FP8 E4M3 | 200 | 6,912,212,992 |
| BF16 protected tensors | 760 | 2,499,026,160 |
| Total | 960 | 9,411,239,152 |
model.safetensors is 11,910,387,016 bytes, the same serialized tensor-file size as the pinned source FP8 model.
A tensor-by-tensor comparison against the pinned source checkpoint found:
| Check | Result |
|---|---|
| Tensor key sets | Identical |
| Matching shapes and stored dtypes | 960 / 960 |
| Bit-identical unchanged tensors | 876 |
| Changed model weights | 42 |
| Changed FP8 scales paired with those weights | 42 |
| Other changed tensors | 0 |
Every changed model weight is within a Heretic target:
mlp.down_projself_attn.o_projlinear_attn.out_projProtected BF16 islands and unrelated quantized tensors remain byte-identical to the source. This bounds the release as a narrow derivative of the pinned FP8 checkpoint rather than an uncontrolled whole-model conversion.
The final repository tree was loaded directly with AutoModelForImageTextToText.from_pretrained, without PEFT and without access to the base-model directory.
Verified properties:
Qwen3_5ForConditionalGenerationcompressed0The capital of France is ParisA small deterministic, greedy-decoding smoke suite passed 6/6 cases:
| Category | Passed |
|---|---|
| Coherence and instruction following | 2 / 2 |
| Tool selection and argument construction | 3 / 3 |
| Simulated tool-result round trip | 1 / 1 |
Observed native tool-call outputs included:
get_weather(city="Chicago", unit="fahrenheit")calculator(expression="17 * 23")geocode(place="Eiffel Tower") before requesting weatherAfter receiving a simulated weather response, the model incorporated its temperature, condition, and observation time into a natural-language answer without issuing another tool call.
This is a smoke test, not a comprehensive agent benchmark. Multi-step real-tool execution, parallel calls, malformed tool results, tool restraint, long-context behavior, multimodal quality, factuality, and broad safety behavior remain unevaluated.
Use recent releases of Transformers, Accelerate, and compressed-tensors with Qwen3.5 support.
python
from transformers import AutoModelForImageTextToText, AutoProcessorrepo_id = "cbert33/Ornith-1.5-9B-FP8-Heretic"model = AutoModelForImageTextToText.from_pretrained(repo_id,dtype="auto",device_map="auto",)processor = AutoProcessor.from_pretrained(repo_id)
python
from transformers import AutoTokenizerrepo_id = "cbert33/Ornith-1.5-9B-FP8-Heretic"tokenizer = AutoTokenizer.from_pretrained(repo_id)messages = [{"role": "user", "content": "Explain why the sky appears blue in two sentences."}]prompt = tokenizer.apply_chat_template(messages,tokenize=False,add_generation_prompt=True,enable_thinking=False,)inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs,max_new_tokens=128,do_sample=False,)print(tokenizer.decode(outputs[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))
The included chat template accepts OpenAI-style function definitions through its tools argument and emits Qwen's native XML tool-call representation.
python
tools = [{"type": "function","function": {"name": "get_weather","description": "Get current weather for a city.","parameters": {"type": "object","properties": {"city": {"type": "string"},"unit": {"type": "string","enum": ["celsius", "fahrenheit"],},},"required": ["city", "unit"],},},}]messages = [{"role": "user","content": "What is the current weather in Chicago in Fahrenheit?",}]prompt = tokenizer.apply_chat_template(messages,tools=tools,tokenize=False,add_generation_prompt=True,enable_thinking=False,)
Your inference framework is responsible for parsing the generated tool call, executing the function, and returning the tool result to the model.
2.13.0+cu1305.6.20.18.00.13.0In the tested direct Transformers path, compressed-tensors materialized/decompressed weights during loading and peak CUDA allocation reached approximately 19.0 GB. The model completed inference on the 16 GB RTX 5080 through WSL unified memory, but generation was slow. An FP8-aware serving runtime should be qualified separately before production deployment.
The repository includes SHA256SUMS. The model tensor hash is:
text
8650b2f6cf35afb133c7225c65b25483211ccbf72901fe9cdd7803c64c32c0af model.safetensors
The source checkpoint is published under the MIT license. This derivative is distributed under the same license. Users must also comply with the licenses and terms applicable to upstream model components and dependencies.