Qwen3.8-27B-abliterated-cyber-preview API & Inference Endpoint

Base

hotdogs/Qwen3.8-27B-abliterated (λ = 1.2), a 27B native vision-language hybrid (full-attention + GDN linear-attention) with a frozen MTP head.

LoRA

Dataset: hotdogs/cyber-sft-agent-qwen38 — 8,400 rows (train 7,140 / valid 840 / test 420). 30 % tool-call, 30 % multi-turn, think-tags 100 % balanced, 22 tools (nmap, sqlmap, metasploit, hydra, crackmapexec, linpeas, wpscan, whatweb, ffuf, gobuster, …), 1,995 unique IP contexts, 0 duplicates.
Config: r=32, alpha=64, dropout=0, ctx=8192, LR=1e-4 (cosine), BF16 (required by GDN hybrid), batch 2 × grad-accum 2 (effective 4), 2 epochs. Trainable 233M (0.85 %).
Target modules: standard q/k/v/o/gate/up/down + GDN in_proj_qkv/out_proj/in_proj_z/a/b.

Merge

Full LoRA (incl. GDN) merged into the base via PEFT merge_and_unload() at default scale (alpha/r = 2.0, ≡ llama.cpp --lora-scaled :1.0).
MTP head recovered after merge: merge_and_unload() + save_pretrained() drops the mtp.* tensors, so the 15 MTP tensors (849 MB) were copied back from the base into the merged safetensors (the LoRA never touched mtp.*, so these weights are byte-correct). Final model = 1,199 tensors, verified loading via unsloth, MTP physically present.
GGUF: convert_hf_to_gguf.py --outtype bf16 (MTP kept in-file), 54.6 GB. Serves with --spec-type draft-mtp.

Evaluation

1. Overfitting check (held-out valid loss)

Table with columns: metric, value
metric	value
valid token loss (840 rows)	0.480
train token loss (300-row sample)	0.384
gap (valid − train)	0.095
valid % loss < 0.2	30.0 %
train % loss < 0.2	27.3 %

A small gap (≤ 0.15) means the model generalizes rather than memorizes. The training loss plateau (~0.44 median, only ~3.8 % of points near zero) is a healthy learning floor for a diverse 8.4K dataset, not overfitting.

2. Scale sweep (llama.cpp `--lora-scaled`, no merge needed)

6 tool prompts + 7 base prompts, temperature 0, identical config across scales:

Table with columns: scale, tool-call format (6), correct real tool (6), base capability (7)
scale	tool-call format (6)	correct real tool (6)	base capability (7)
baseline (no LoRA)	0/6	0/6	6/7
0.25	1/6	0/6	7/7
0.5	6/6	2/6	7/7
0.75	6/6	4/6	7/7

Scale 1.0 is the sweet spot: every tool prompt emits a well-formed <tool_call> and picks the correct real tool, while base capability stays at 7/7. Damping below 1.0 hurts tool selection (hallucinated tools like web_fetch, wp_recon); going above ~2.0 degrades parameter quality (e.g. ports="-s -p 1-65535"). The merge therefore uses the default alpha/r = 2.0, which corresponds to this optimal 1.0 point.

3. KL divergence (base ‖ merged)

Measured on a shared continuation that the merged model sampled, comparing top-25 next-token log-probs from the base vs the merged GGUF (both via llama.cpp llama-server):

Table with columns: prompt, KL (base ‖ merged)
prompt	KL (base ‖ merged)
17 × 43	0.0066
Python reverse-string	0.1559
Capital of France	0.0206
stack vs queue	0.0556
15 % of 200	0.0030
boiling point	0.0041
mean (base prompts)	0.0410
port-scan tool prompt	0.7332
SQLi tool prompt

The ~20× gap between base-prompts (0.04) and tool-prompts (0.81) is the quantitative signature of a surgically targeted LoRA: it re-wires tool-calling behavior without disturbing general knowledge. This matches the abliterated base's own near-zero first-token KL (0.0001) — the merge adds capability, not drift.

4. MTP (Multi-Token Prediction)

Table with columns: metric, value
metric	value
MTP draft acceptance rate	0.77 (51 / 66)
mean draft length	2.55
head location	in-file (bf16 GGUF)

Well above the ~0.3 threshold where speculative decoding pays off — MTP is active and beneficial.

Files

Table with columns: file, purpose
file	purpose
`model-00001-of-00002.safetensors` / `model-00002-of-00002.safetensors`	merged weights (MTP in shard 2)
`config.json`	`Qwen3_5ForConditionalGeneration`
`chat_template.jinja` / `tokenizer.json` / `tokenizer_config.json` / `processor_config.json`	Qwen3.5 chat + vision template

Disclaimer

This is a preview built for authorized security research and red-teaming. It will not refuse and may emit instructions for exploiting systems. Use only on systems you own or are explicitly authorized to test. Not for misuse/harmful activity. Apache-2.0, inherited from the base.

Base

hotdogs/Qwen3.8-27B-abliterated (λ = 1.2), a 27B native vision-language hybrid (full-attention + GDN linear-attention) with a frozen MTP head.

LoRA

Dataset: hotdogs/cyber-sft-agent-qwen38 — 8,400 rows (train 7,140 / valid 840 / test 420). 30 % tool-call, 30 % multi-turn, think-tags 100 % balanced, 22 tools (nmap, sqlmap, metasploit, hydra, crackmapexec, linpeas, wpscan, whatweb, ffuf, gobuster, …), 1,995 unique IP contexts, 0 duplicates.
Config: r=32, alpha=64, dropout=0, ctx=8192, LR=1e-4 (cosine), BF16 (required by GDN hybrid), batch 2 × grad-accum 2 (effective 4), 2 epochs. Trainable 233M (0.85 %).
Target modules: standard q/k/v/o/gate/up/down + GDN in_proj_qkv/out_proj/in_proj_z/a/b.

Merge

Full LoRA (incl. GDN) merged into the base via PEFT merge_and_unload() at default scale (alpha/r = 2.0, ≡ llama.cpp --lora-scaled :1.0).
MTP head recovered after merge: merge_and_unload() + save_pretrained() drops the mtp.* tensors, so the 15 MTP tensors (849 MB) were copied back from the base into the merged safetensors (the LoRA never touched mtp.*, so these weights are byte-correct). Final model = 1,199 tensors, verified loading via unsloth, MTP physically present.
GGUF: convert_hf_to_gguf.py --outtype bf16 (MTP kept in-file), 54.6 GB. Serves with --spec-type draft-mtp.

Evaluation

1. Overfitting check (held-out valid loss)

Table with columns: metric, value
metric	value
valid token loss (840 rows)	0.480
train token loss (300-row sample)	0.384
gap (valid − train)	0.095
valid % loss < 0.2	30.0 %
train % loss < 0.2	27.3 %

2. Scale sweep (llama.cpp `--lora-scaled`, no merge needed)

6 tool prompts + 7 base prompts, temperature 0, identical config across scales:

Table with columns: scale, tool-call format (6), correct real tool (6), base capability (7)
scale	tool-call format (6)	correct real tool (6)	base capability (7)
baseline (no LoRA)	0/6	0/6	6/7
0.25	1/6	0/6	7/7
0.5	6/6	2/6	7/7
0.75	6/6	4/6	7/7

3. KL divergence (base ‖ merged)

Measured on a shared continuation that the merged model sampled, comparing top-25 next-token log-probs from the base vs the merged GGUF (both via llama.cpp llama-server):

Table with columns: prompt, KL (base ‖ merged)
prompt	KL (base ‖ merged)
17 × 43	0.0066
Python reverse-string	0.1559
Capital of France	0.0206
stack vs queue	0.0556
15 % of 200	0.0030
boiling point	0.0041
mean (base prompts)	0.0410
port-scan tool prompt	0.7332
SQLi tool prompt

4. MTP (Multi-Token Prediction)

Table with columns: metric, value
metric	value
MTP draft acceptance rate	0.77 (51 / 66)
mean draft length	2.55
head location	in-file (bf16 GGUF)

Well above the ~0.3 threshold where speculative decoding pays off — MTP is active and beneficial.

Files

Table with columns: file, purpose
file	purpose
`model-00001-of-00002.safetensors` / `model-00002-of-00002.safetensors`	merged weights (MTP in shard 2)
`config.json`	`Qwen3_5ForConditionalGeneration`
`chat_template.jinja` / `tokenizer.json` / `tokenizer_config.json` / `processor_config.json`	Qwen3.5 chat + vision template

Qwen3.8-27B-abliterated-cyber-preview

README

Base

LoRA

Merge

Evaluation

1. Overfitting check (held-out valid loss)

2. Scale sweep (llama.cpp `--lora-scaled`, no merge needed)

3. KL divergence (base ‖ merged)

4. MTP (Multi-Token Prediction)

Files

Disclaimer

Explore FriendliAI today

README

Base

LoRA

Merge

Evaluation

1. Overfitting check (held-out valid loss)

2. Scale sweep (llama.cpp `--lora-scaled`, no merge needed)

3. KL divergence (base ‖ merged)

4. MTP (Multi-Token Prediction)

Files

Disclaimer

Qwen3.8-27B-abliterated-cyber-preview

README

Base

LoRA

Merge

Evaluation

1. Overfitting check (held-out valid loss)

2. Scale sweep (llama.cpp --lora-scaled, no merge needed)

3. KL divergence (base ‖ merged)

4. MTP (Multi-Token Prediction)

Files

Disclaimer

Explore FriendliAI today

README

Base

LoRA

Merge

Evaluation

1. Overfitting check (held-out valid loss)

2. Scale sweep (llama.cpp --lora-scaled, no merge needed)

3. KL divergence (base ‖ merged)

4. MTP (Multi-Token Prediction)

Files

Disclaimer

2. Scale sweep (llama.cpp `--lora-scaled`, no merge needed)

2. Scale sweep (llama.cpp `--lora-scaled`, no merge needed)