Extraction and rank
The complete checkpoint audit compared all 1045 indexed tensors: 123 changed, 922 numerically unchanged. The default adapter covers 80 ordinary modules and 40 routed expert parameter groups, totaling 10,320 individual active matrices. Each expert is factorized separately. Saved FP32 factors have physical rank 1 and scaling 1, totaling 106.287 MB.
Measured retained active delta energy: 99.569570%. Minimum individual matrix: 99.290751%. Energy measures weight-space approximation, not model quality.
Table with columns: Rank, Weighted captured energy, Minimum matrix captured energy| Rank | Weighted captured energy | Minimum matrix captured energy |
|---|
| 1 | 99.569570% | 99.290751% |
| 2 | 99.577011% | 99.305606% |
| 4 | 99.585539% | 99.319760% |
| 8 | 99.598830% | 99.342985% |
Rank policy: smallest candidate with at least 99% weighted and 95% minimum individual matrix energy; fallback 8 is explicitly marked if the threshold fails. Dense extraction uses an FP64 Gaussian range finder (16 columns, three subspace iterations), with independent exact FP64 SVD checks. The subsequent compression uses the native LoRAForge FlashTSQR tensor API (truncate_lora); backend records are in extraction_report.json. Library revision: b2517ebe9be14bacce18ec4fef19774e7772c0fb. Per-expert packing uses an experimental PEFT target_parameters bridge, rather than the linear-only LoRAForge file API. No accelerated MoE multi-adapter serving claim is made.
The checkpoint audit found 3 changed MTP tensor groups. Their compressed factors are supplied separately in optional_mtp_adapter.safetensors, and are not applied by the default PEFT adapter. Transformers ignores mtp.*; MTP draft integration and speculative decoding are untested.
Actual loading and generation validation
The full BF16 edited checkpoint was compared with the full pinned BF16 base plus the unmerged PEFT adapter, on NVIDIA H100 NVL. All saved adapter factor tensors reloaded exactly, including the expert packing and scale. Six benign prompts used greedy decoding, thinking disabled, and at most 96 generated tokens each; 3/6 generated texts were identical. See validation.json for exact prompts, outputs, and full-vocabulary first-token KL values. Long answers may be cut off at the token limit. This is a loading and short-generation smoke test, not a comprehensive capability, refusal, multimodal, or MTP benchmark.
Tested: Transformers 5.22.1, PEFT 0.21.2, PyTorch 2.11.0+cu130. The parent needs approximately 72 GB for BF16 weights, plus runtime memory. A 94 GB H100 NVL was used.
Loading
Authenticate with Hugging Face if required by the upstream base or source tokenizer. Use a PEFT version supporting target_parameters for MoE experts. load_adapter.py pins the base revision and uses the source tokenizer/chat template:
python load_adapter.py loraforge/huihui-qwen3.6-35b-a3b-abliterated-r1
import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizerfrom peft import PeftModel adapter = "loraforge/huihui-qwen3.6-35b-a3b-abliterated-r1"base = AutoModelForImageTextToText.from_pretrained( "Qwen/Qwen3.6-35B-A3B", revision="345d467b117d1fd5eda9b879a5988de9ae5f47a7", dtype=torch.bfloat16, device_map="auto", attn_implementation="sdpa",)model = PeftModel.from_pretrained(base, adapter, is_trainable=False).eval()tokenizer = AutoTokenizer.from_pretrained(adapter)
Attribution and license
The exact base LICENSE (Apache-2.0), upstream model cards, pinned revisions and payload SHA256 receipts are preserved. Credit Qwen for the base and huihui-ai for the edited model. The upstream card links the abliteration approach at remove-refusals-with-transformers. This package reconstructs a checkpoint delta; it does not establish that the source was originally trained using LoRA or that its original training rank equals the retained rank.
Citation
@misc{li2026loraforge, title = {{LoRAForge}: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-{LoRA} Serving}, author = {Shenghui Li and Thiemo Voigt}, year = {2026}, eprint = {2611.01256}, archivePrefix = {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2611.01256}}