ornith-1.5-9b-sft-r16 API & Inference Endpoint

🔬 Model Details

Table with columns: Property, Value
Property	Value
Base Model	`ornith-ai/Ornith-1.5-9B`
Base Architecture	Qwen3.5 Dense (`qwen3_5`), Multimodal Reasoner
PEFT Method	QLoRA (4-bit NF4 quantized base + Float32/BFloat16 LoRA adapters)
LoRA Rank ( $r$ )	16
LoRA Alpha ( $α$ )	32 ( $α / r = 2.0$ )
LoRA Dropout	0.0
Target Modules	Attention & MLP projections (`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`)
Trainable Parameters	29,097,984 / 9,438,911,728 (0.31%)
Context Length	2,048 tokens (Base supports up to 262K)
Training Hardware	NVIDIA Tesla T4 (16 GB VRAM) on Google Cloud
Training Duration	446.1 seconds
Framework	Unsloth 2026.8.22 + TRL + HuggingFace PEFT
Base License	MIT

📊 Training Convergence & Loss Curve

Trained for 3 full epochs (9 optimization steps with effective batch size 8):

Table with columns: Step, Epoch, Training Loss, Learning Rate, Grad Norm
Step	Epoch	Training Loss	Learning Rate	Grad Norm
1/9	0.4	2.614	0.00e+00 (Warmup)	3.655
2/9	0.8	2.683	2.00e-04	4.080
3/9	1.0	2.691	1.92e-04	2.888

Initial Loss: 2.614
Final Converged Loss: 1.105
Mean Loss across Run: 1.806

📦 Training Data Specification

Domain: IEC 61131-3 Structured Text (ST) / Siemens SCL for PLC and DCS industrial automation.
Corpus Structure: OSCAT basic automation library blocks, Plant-01 twin interlocks, and safety refusal prompts.
Format: ChatML / OpenAI JSONL multi-turn format (messages schema).

💻 How to Load and Use with Transformers & PEFT

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "ornith-ai/Ornith-1.5-9B"
adapter_id = "adarrshDev/ornith-1.5-9b-sft-r16"

# 1. Load Tokenizer & Base Model
tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# 2. Attach the Fine-Tuned Ornith LoRA Adapter
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

# 3. Generate Structured Text logic
messages = [
    {"role": "user", "content": "Write an IEC 61131-3 Structured Text FUNCTION_BLOCK for a debounced digital input with configurable ON/OFF delay timers."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

⚖️ License & Governance

This adapter is distributed under the MIT License, consistent with the ornith-ai/Ornith-1.5-9B upstream base model.

Property

Value

Base Model

ornith-ai/Ornith-1.5-9B

Base Architecture

Qwen3.5 Dense (qwen3_5), Multimodal Reasoner

PEFT Method

QLoRA (4-bit NF4 quantized base + Float32/BFloat16 LoRA adapters)

LoRA Rank ( $r$ )

LoRA Alpha ( $α$ )

32 (

α / r = 2.0

)

LoRA Dropout

0.0

Target Modules

Attention & MLP projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)

Trainable Parameters

29,097,984 / 9,438,911,728 (0.31%)

Context Length

2,048 tokens (Base supports up to 262K)

Training Hardware

NVIDIA Tesla T4 (16 GB VRAM) on Google Cloud

Training Duration

446.1 seconds

Framework

Unsloth 2026.8.22 + TRL + HuggingFace PEFT

Base License

MIT

Step

Epoch

Training Loss

Learning Rate

Grad Norm

1/9

0.4

2.614

0.00e+00 (Warmup)

3.655

2/9

0.8

2.683

2.00e-04

4.080

3/9

1.0

2.691

1.92e-04

2.888

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "ornith-ai/Ornith-1.5-9B"
adapter_id = "adarrshDev/ornith-1.5-9b-sft-r16"

# 1. Load Tokenizer & Base Model
tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# 2. Attach the Fine-Tuned Ornith LoRA Adapter
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

# 3. Generate Structured Text logic
messages = [
    {"role": "user", "content": "Write an IEC 61131-3 Structured Text FUNCTION_BLOCK for a debounced digital input with configurable ON/OFF delay timers."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

ornith-1.5-9b-sft-r16

README

🔬 Model Details

📊 Training Convergence & Loss Curve

📦 Training Data Specification

💻 How to Load and Use with Transformers & PEFT

⚖️ License & Governance

Explore FriendliAI today

README

🔬 Model Details

📊 Training Convergence & Loss Curve

📦 Training Data Specification

💻 How to Load and Use with Transformers & PEFT

⚖️ License & Governance