Method and files
The dense difference is formed in FP32 from immutable public checkpoint payloads verified by SHA256. A Gaussian range finder (128 columns,3 subspace iterations,FP64 reorthogonalization/projection) first creates factors without materializing an author adapter. The resulting factors are passed to loraforge.adapters.compress_peft_adapter: actual FlashTSQR QR plus FP64 torch eigendecomposition. The native loraforge_manifest.json is unmodified. Range finding is an extraction pre-step, not a claim that generic QR is FlashTSQR.
adapter_config.json and adapter_model.safetensors are standard PEFT artifacts; use the exact base revision. extraction_report.json records all per-module errors, independent SVD checks and source provenance. validation.json contains real-model loading and short generation checks. Target tokenizer assets are included; token IDs were audited equal to both bases.
Load
Requires Qwen3.5 support in Transformers and compatible PEFT; tested with Transformers5.22.1, PEFT0.21.2, PyTorch2.11.0+cu130 onNVIDIA H100 NVL.
import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizerfrom peft import PeftModel adapter_id = "loraforge/deepseek-v4-pro-qwen3.5-9b-heretic-jackrong-r4"base = AutoModelForImageTextToText.from_pretrained( "Jackrong/DeepSeek-V4-Pro-Qwen3.5-9B", revision="7d7edca1b1e820d89127f9c8ac4b035e0f27652e", dtype=torch.bfloat16, device_map="auto",)model = PeftModel.from_pretrained(base, adapter_id).eval()tokenizer = AutoTokenizer.from_pretrained(adapter_id, fix_mistral_regex=True)
Alternatively download this repository and run python load_adapter.py loraforge/deepseek-v4-pro-qwen3.5-9b-heretic-jackrong-r4. The model is multimodal Qwen3.5; the smoke check exercises text generation only. The tokenizer assets match the fixed target version. No adapter merge is required.
One-dimensional precision differences
The target also has24 changed norm vectors from native storage-precision conversion (total squared delta norm0.00770741014). These are not included in the linear LoRA, keeping the PEFT file compatible with LoRAForge's linear-only contract. optional_precision_overrides.safetensors stores their absolute target values. It is not applied automatically by PEFT or LoRAForge; the helper's --precision-overrides flag applies it explicitly. Validation below used the standard adapter without this optional patch.
Validation and limits
All exported factor tensors were checked equal after real PEFT loading; each module has physical rank4 and unit scale. Reconstruction errors were checked against the dense delta and previous independent spectral intervals. A representative modified4096×4096 attention projection was also checked using full FP64 SVD.
Table with columns: Neutral prompt, Next-token KL, full target → adapter, Greedy first token matches| Neutral prompt | Next-token KL, full target → adapter | Greedy first token matches |
|---|
| Compute 17 * 23. Give only the number. | 0.000135996 | True |
| Explain in two sentences why the sky looks blue. | 6.3459e-05 | True |
| Write a short Python function that returns the largest value in a nonempty list. | 4.32829e-06 | True |
These three short greedy continuations are a loading/smoke check, not an overall capability or safety benchmark. Weight energy is not a performance guarantee. Historical exact training-parent provenance is not independently proven; this repository precisely identifies the public revisions compared. BF16 base loading and merging introduce additional numerical effects beyond the reported FP32 weight approximation. No inference speedup is claimed by this release.
Attribution and license
Original Qwen, Jackrong and xtools-at model authors retain attribution for their respective contributions. Source model cards are preserved under provenance/. Published upstream declarations are Apache-2.0; this redistribution includes LICENSE, NOTICE and an explicit record of modifications. LoRAForge performed reconstruction and compression, not the original training. This repository does not imply endorsement by upstream authors.
Citation
@misc{li2026loraforge, title = {{LoRAForge}: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-{LoRA} Serving}, author = {Shenghui Li and Thiemo Voigt}, year = {2026}, eprint = {2611.01256}, archivePrefix = {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2611.01256}}