Layer-delta analysis (what was measured before merging)
Before merging, the two fine-tunes were compared tensor-by-tensor against their common base in fp32, streamed from safetensors with no full-model load. For every tensor t: ΔD = D(t) − base(t), ΔS = S(t) − base(t), then tensors were grouped (per text layer 000–063, vision groups, embeddings+lm_head, mtp/other) and aggregated.
1. Structural validation. All three models share an identical 1184-tensor skeleton; a strict check of every tensor's dtype and shape ran first (abort on any shape mismatch — none found). The only structural asymmetry was dtype-only (bf16 vs fp32 scale-like tensors), and a small set of tensors present only in DaisyRose was inventoried and handled separately.
2. The two fine-tunes moved in essentially orthogonal directions. The cosine similarity between ΔD and ΔS is ≈ 0 for nearly every layer (|cos| < 0.003 for all 64 text layers; e.g. embeddings_and_lm_head cos = −0.0001). The two fine-tunes are not two points along one training path — they are truly divergent soups, which is exactly the regime where parameter averaging adds rather than cancels knowledge.
3. Delta magnitudes by depth. Both fine-tunes drift more in mid-depth layers: |ΔS| grows from 0.44 (embeddings) to a peak of ~0.51 around layers 24–33, then decays to ~0.26 at the deepest layers; |ΔD| follows the same shape with smaller amplitude (peak ~0.37 at layers 16–26, → 0.28–0.30 late). sabi-noema is the stronger mover overall (norm ~0.44–0.50 vs ~0.34–0.37 for DaisyRose).
4. The 50/50 average is variance-preserving here. For orthogonal deltas, the merged deviation norm is |ΔD + ΔS|/2 = ½·sqrt(|ΔD|² + |ΔS|²) ≈ 0.30 per layer — i.e. the merge retains ~100% of the combined fine-tune signal (uniform-weight averaging never loses combined signal for orthogonal deltas; it loses up to 50% only when deltas are parallel). No layer displayed the "harmful" regime (large positive cosine), so no per-layer reweighting was applied — plain linear averaging everywhere.
Merge procedure
- Streaming merge (one tensor resident at a time), fp32 arithmetic, output cast back to bf16:
out = (D(t) + S(t)) / 2.
- Identical-tensor shortcut: the entire vision tower is bit-identical between the two fine-tune deltas (ΔD = ΔS = 0) — vision weights pass through unchanged.
- Byte-balanced sharding into 3 safetensors shards + regenerated index; 1184/1184 tensors verified post-merge.
- No permutation/alignment issues: both fine-tunes share exact tensor naming and ordering, so the delta analysis doubles as a merge alignment check.
NVFP4 quantization (this repo)
RadixArk/DaisyRose-style recipe, executed with llmcompressor 0.15.0 oneshot:
Table with columns: Setting, Value| Setting | Value |
|---|
| Scheme | NVFP4 (W4A4), group size 16, tensor_group strategy |
| Weight observer | memoryless minmax, fp8 e4m3 scales |
| Activation quant | W4A4, static scales collected during calibration |
| Calibration | 512 samples × 2048 ctx from a local chat+code jsonl mix (~525-sample set) |
| Sequential pipeline | layer-by-layer streaming to GPU, CPU offload — never needs full 52 GB in VRAM |
| Keep in bf16 | lm_head + full vision tower (model.visual.*) |
| Output | 19 GB, nvfp4-pack-quantized format |
Result: ~2.9× smaller than the bf16 merge, with native 4-bit matmul on Blackwell. Verified with a vLLM generation smoke test (loglikelihood + CoT gates) before upload.
Usage (vLLM)
vllm serve vince62s/Qwen3.81-27B-NVFP4 \ --served-model-name qwen3.81 \ --reasoning-parser qwen3 \ --max-model-len 32768 # scale to available KV; 49k fits a 32GB-5090 at util 0.88
- Chat template follows the Qwen3.5 thinking format;
enable_thinking defaults to on with xhigh reasoning effort.
- Sampling (
generation_config.json): temperature: 1.0, top_p: 0.95, top_k: 20.
- Multimodal:
image_token_id 248056, architecture Qwen3_5ForConditionalGeneration.
Table with columns: Benchmark, Setting, Result| Benchmark | Setting | Result |
|---|
| GPQA Diamond (mirror, 198 q) | lm-eval 0.4.13, loglikelihood | 43.9% |
| GPQA Diamond (CoT gate, 10 q) | generative, temp 0.6 | 9/10 |
| GPQA Diamond (CoT gate, 10 q) | generative, temp 1.0 (server sampling) | 8/10 |
| Terminal-Bench 2.1 (subset, pass@1) | harbor + terminus-2, 3× task timeouts | ~8/10 early batches; full 89-task suite in progress |
Terminal-Bench notes for reproducibility: Terminus-2 must be told the true context window via --ak model_info={"max_input_tokens":40000,"max_output_tokens":8192} — otherwise it assumes ~1M tokens, overflows a 49k server, and wedges in a summarization retry loop. With the fix, episodes self-trim and complete.
Intent
Qwen3.81 combines DaisyRose-27B's agentic/terminal strengths with sabi-noema's hybrid-attention reasoning base in one open 27B checkpoint, published in both bf16 and NVFP4. Analysis scripts (analyze_deltas.py, merge_models.py, quantize_nvfp4.py) replicate every step above with no additional tooling beyond torch, safetensors and llmcompressor.