DanielTobi0

qwen3.8-27b-merged

README

License: apache-2.0

Loading

python
import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

REPO = "DanielTobi0/qwen3.8-27b-merged"

model = AutoModelForImageTextToText.from_pretrained(
    REPO,
    dtype = torch.bfloat16,
    device_map = "auto",
)
model.eval()

tokenizer = AutoTokenizer.from_pretrained(REPO)

This is a native vision-language model (Qwen3_5ForConditionalGeneration), so it loads through AutoModelForImageTextToText. Use AutoModelForCausalLM instead only if you want the text-only decoder (Qwen3_5ForCausalLM) and intend to drop the vision tower. The weights are ~55 GB in bf16, so plan for an 80 GB accelerator, or pass a quantization_config to fit something smaller.

Qwen3.8's gated-deltanet layers have a fast Triton path that transformers does not ship. Without it you will see The fast path is not available ... Falling back to torch implementation and noticeably slower inference. Install flash-linear-attention and causal-conv1d, or load through Unsloth, which bundles them.

Generating

python
messages = [
    {"role": "system", "content": "You are Crowther AI. Respond authoritatively, precisely, and formally using 'we' without contractions or em dashes."},
    {"role": "user",   "content": "Which layer of the sovereign intelligence stack is partnered rather than owned?"},
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize = False,
    add_generation_prompt = True,
    reasoning_effort = "xhigh",      # must match training
)

inputs = tokenizer(text, return_tensors = "pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens = 1024, do_sample = False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens = True))

Reading the output

add_generation_prompt=True ends the prompt with <|im_start|>assistant\n<think>\n, so generation begins already inside the thinking block. The model emits its reasoning, closes it with </think>, then writes the answer. Split on the closing tag to separate them:

python
reasoning, _, answer = decoded.partition("</think>")

Set enable_thinking=False in apply_chat_template to suppress reasoning, but note the model was trained exclusively on thinking-enabled examples.

Serving

The merged weights are a standard qwen3_5 checkpoint, so any runtime with Qwen3.8 support can serve them — for example:

bash
vllm serve DanielTobi0/qwen3.8-27b-merged --dtype bfloat16

Pass the reasoning effort through your client's chat-template arguments (in an OpenAI-compatible request, chat_template_kwargs: {"reasoning_effort": "xhigh"}) so the system prefix matches training. Serving configuration was not exercised during this fine-tune — verify the rendered prompt before relying on it.

Training

Table

Base model	`unsloth/Qwen3.8-27B` (27B, hybrid gated-deltanet + gated attention, VLM)
Dataset	`DanielTobi0/qwen3_finetuning_dataset` — 236 samples
Method	LoRA (SFT) merged to 16-bit, loss on assistant turns only
Trainable params	108,789,760 of 27,465,518,320 (0.40%)
Rank / alpha / dropout	16 / 16 / 0
Sequence length	2048 (longest sample: 1,822 tokens — no truncation)
Epochs / steps	3 / 90

Adapted modules

Qwen3.8-27B interleaves two attention types, so the fine-tune covered both rather than the usual seven names:

q_proj, k_proj, v_proj, o_proj — the 16 full gated-attention layers
in_proj_qkv, in_proj_z, out_proj — the 48 gated-deltanet linear-attention layers
gate_proj, up_proj, down_proj — the MLP in all 64 layers

Targeting only the standard four attention names would leave 48 of 64 layers unadapted.

Limitations

The training set is small (236 samples) and was trained for 3 epochs at 2e-4, which is an aggressive schedule for a 27B model at this data scale. The model reliably reproduces the target register and reasoning format, but it does not reliably reproduce the factual content of the training data — spot checks show it confabulating specifics that contradict its own training examples. Treat it as a style and format fine-tune, not a knowledge base, and ground factual claims externally.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

DanielTobi0

Model Tree

Base

unsloth/Qwen3.8-27B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality