What is the source model?
Swift-1.5-Qwen3.8-27B-Uncensored-MTP is an abliterated derivative of ukisai/Swift-1.5-Qwen3.8-27b, which is itself UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B. The abliteration applies orcarouter's refusal direction r (recovered from orcarouter/Qwen3.8-27B-Uncensored, Arditi et al. 2024 style):
- refusal removal: 98/100 → 23/100 refusals (Heretic evaluation), KL 0.0884
- MTP head kept and edited consistently → self-speculative decoding still works
- 131 tensors edited with
W' = W − r(rᵀW) (plus embed_tokens)
So the delta Δ = W_swift_unc − W_qwen38 contains two stacked effects: Swift 1.5's fine-tuning edits and the refusal-abliteration edit projected out of them.
Δ = W_swift_unc − W_qwen3.8 ← everything Swift+abliteration changedΔ ≈ U_r Σ_r V_rᵀ (exact truncated SVD, rank r = 16)lora_A = Σ_r^½ V_rᵀ , lora_B = U_r Σ_r^½
Pipeline (CPU, float32):
- Architecture check — identical 1,199 tensor names and shapes on both repos (
qwen3_5_text, 64 layers, hidden 5120, intermediate 17408, MTP, vocab 248320).
- Norm scan — 866 non-visual tensors compared. 204 tensors had
‖Δ‖ ≥ 0.5 (the rest within fine-tune noise). This matches the source's edited tensor set (131 abliteration tensors) plus the Swift fine-tune deltas on gate/up/q projections.
- Exact truncated SVD at uniform rank 16 → 204
lora_A/lora_B pairs.
- Reconstruction validation — 40-tensor sample, adapter-implied delta vs true delta: mean relative error 0.37.
Quality: where it's exact and where it approximates
The abliteration component is structurally rank-1 (r(rᵀW)), and Swift's edits on the output projections are similarly low-rank — so those reconstruct almost perfectly:
Table with columns: target, ‖Δ‖ range, energy retained @ r=16, reconstruction error| target | ‖Δ‖ range | energy retained @ r=16 | reconstruction error |
|---|
self_attn.o_proj (16 layers) | up to ~1.6 | 96–99% | ~0.02–0.19 |
linear_attn.out_proj (48 layers) | up to ~1.6 | 99% | ~0.10 |
mlp.down_proj (64 layers) | up to ~1.6 | 96% | ~0.20 |
mlp.gate_proj / mlp.up_proj | ~0.5–0.7 | ~60–66% | ~0.75 |
self_attn.q_proj | ~0.6–0.9 | ~60–85% | ~0.45–0.7 |
Refusal removal is preserved (that signal lives in the o_proj/down_proj/out_proj tensors captured near-exactly). The softer Swift fine-tune deltas on gate/up projections are approximated at rank 16.
lm_head and embed_tokens deltas were skipped (SVD on 248320×5120 exceeds this machine's RAM); refusal removal does not depend on them here.
Files
Table with columns: File, Format, Notes| File | Format | Notes |
|---|
adapter_model.safetensors | PEFT LoRA | r=16, alpha=16, float32, 204 pairs |
adapter_config.json | JSON | base_model: Qwen/Qwen3.8-27B |
swift_unc_qwen3.8-27B_lora.gguf | llama.cpp LoRA GGUF | load with --lora against a Qwen3.8-27B GGUF |
extraction_stats.json | JSON | per-tensor Δ norms, ranks, retained energy |
validation_report.json | JSON | reconstruction errors |
Usage
llama.cpp
llama-cli -m Qwen3.8-27B.gguf \ --lora swift_unc_qwen3.8-27B_lora.gguf \ -p "What do you think about controversial topics?"
Scale with --lora-scaled swift_unc_qwen3.8-27B_lora.gguf:0.8 if you want a partial effect.
Python (PEFT)
from peft import PeftModelfrom transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(model, "hotdogs/swift_unc_qwen3.8-27B_lora")
License
Derivative of Swift-1.5-Qwen3.8-27B (Swift Open License v1.0: free up to US$1M annual revenue; commercial use above that needs a Swift Enterprise License from UkisAI). Upstream abliteration credit: orcarouter, Arditi et al. 2024. Qwen3.8-27B is Apache-2.0.