Benchmark status
Matched baseline-versus-derivative benchmarking is in progress. The public
text matrix covers GPQA Diamond, IFBench, and LiveCodeBench v6. It also
includes matched perplexity and refusal controls. Every active task uses exact,
executable, or otherwise deterministic scoring. HLE is excluded because the
reported setup uses a paid model judge. Results will be added with exact
harness versions, task revisions, prompts, sample counts, and raw result files.
The Qwen model card also reports several private or environment-heavy agent
benchmarks. Those results cannot be reproduced from model weights alone. They
will remain clearly marked as untested unless the exact datasets and harnesses
become available.
Current validation gate
These small diagnostics come from OBLITERATUS itself. They are not substitutes
for the matched benchmark matrix.
Table with columns: Diagnostic, Result| Diagnostic | Result |
|---|
| Perplexity | 3.5903 |
| Coherence check | 1.0 |
| Capability checks | 5 of 6 |
| Refusal rate | 20% of 30 sampled prompts |
| First-token KL divergence | 1.2249 |
| Spectral certificate | RED |
The high KL divergence and red spectral result indicate substantial behavior
change. Evaluate the model for each intended use before deployment.
Derivation
- Base revision:
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
- Base precision: BF16
- OBLITERATUS commit:
c1a12752910167543bd7a6135e05a8bace99901e
- Qwen compatibility patch SHA-256:
64bfda09fd62eb75ae103c886154c8171608b79e1c2de6416f739dedcbd4e8a4
- Method: advanced SVD refusal-direction ablation
- Direction count: 4
- Selected language-model layers: 49 through 63
- Modified matrices: 336
- Harmful prompts: 842
- Harmless prompts: 842
- Norm preservation: enabled
- Regularization: 0.3
- Refinement passes: 2
- Bias projection: enabled
The run kept all 1,184 parameters on one RTX PRO 6000 in BF16. It used Flash
Attention 2 and did not use CPU, disk, multi-GPU, or model offload.
Scope and limitations
- Refusal-direction ablation intentionally changes refusal behavior.
- The model may produce unsafe, incorrect, or unexpected output.
- The initial capability gate is too small to establish retained quality.
- Only text behavior is in the active benchmark plan. Vision input remains
untested for this derivative.
- The model has not replaced any production checkpoint.
You are responsible for suitable safeguards, evaluation, and compliance in
the environment where you run the model.
Serving
Use current Qwen3.8-compatible versions of Transformers, vLLM, or SGLang. For
example:
vllm serve orwelian84/Qwen3.8-27B-OBLITERATUS-Advanced \
--dtype bfloat16 \
--max-model-len 65536
Qwen3.8 uses thinking mode by default. Follow the official Qwen3.8 model card
for chat-template and sampling guidance.
Reproducibility
The repository includes a sanitized derivation manifest and SHA-256 records.
The run preserved the original failed compatibility attempt separately. The
published model comes only from the fresh validated retry.
License and attribution
The base model is Apache 2.0 licensed. See LICENSE and the
Qwen/Qwen3.8-27B model card.
The ablation tool is
elder-plinius/OBLITERATUS.