RoelV

Qwopus3.6-27B-v2-oQ6-fp16-mtp

README

License: apache-2.0

💡 1. Base Model, Training Library & Cooperation

[!TIP] Vision & Tool Calling Support: Qwopus3.6-27B-v2 natively supports vision and tool-use capabilities. To enable vision functionality, download mmproj.gguf from the GGUF Repository and place it in the same directory as the main .gguf file.

[!WARNING] Community Release Notice: Qwopus3.6-27B-v2 is an experimental community release and has not undergone complete safety evaluations or standard benchmarking. It is intended solely for research and exploration.

📖 2. Background & Motivation

⚡ 3. Reasoning Efficiency & MTP Speedup

📊 4. Evaluation & Benchmarks

🗺️ 5. Training & Data Pipeline Overview

The training process fuses Trace Inversion data augmentation with a Three-Stage Curriculum Learning pipeline. The core engineering focuses on expanding context length gradually while training on reconstructed reasoning traces to guarantee format stability.

text
[ 🗺️ Trace Inversion: Reconstructing Distillation Workflow ]

  A. Surrogate Model Training (Trace Inverter)
     Open-source Model (GLM-5.1 / DS-V4) ──► Complete Reasoning Chain ──► [ Qwen3-235B Compression ] ──► Reasoning Bubbles
                                              │                                   │
                                              └──────────► [ Training ] ◄─────────┘
                                                   (Base: Qwen3-4B-Instruct)
                                                   (Result: Trace-Inverter-4B)

  B. Inversion Phase: Reconstructing Claude-4.7-Max
     _______________________________________________________
    |                                                       |
    |  Claude-4.7-Max API ──► Compressed Bubbles + Answer   |
    |_______________________________________________________|
                      │
                      ▼
    [ 🧠 Trace-Inverter-4B (Logic Reconstructor) ] ──► Synthetic Deep Reasoning Trace (Learnable CoT)
                      │
                      ▼
    [ 🧩 Data Splicing ] ◄────────── (Original Prompt + Response)
    (Embed reconstructed CoT in <think> tags, splicing with original prompt/response)
                      │
                      ▼
             (Result: claude-opus-4.6/4.7 inverted sets)

  C. Final SFT Curriculum Pipeline
     ___________________________________________
    |                                           |
    |          Base Model (Qwen3.6-27B)         |
    |___________________________________________|
                      │
                      ▼
    [ 📦 Phase 1: Format Inception ] ──► [ 🛠️ Phase 2: Complexity Expansion ] ──► [ 🚀 Phase 3: Long-Context SFT ]
      ( < 4096 tokens )                     ( 4096 - 8192 tokens )                 ( 8192 - 32K tokens )
      (Short-context stable format)         (Medium-complexity reasoning)          (Long/Multi-turn / 10% replay)
                      │                                                                       │
                      └─────────────────────────────┬─────────────────────────────────────────┘
                                                    ▼
                                   _____________________________________________
                                  |                                             |
                                  |   🌟 Final Model: Qwopus3.6-27B-v2          |
                                  |_____________________________________________|

🎯 6. Three-Stage Curriculum Learning

To steadily scale up the reasoning quality under long-context inference, Qwopus3.6-27B-v2 adopts a Curriculum Learning strategy, progressively mixing longer and more complex reasoning templates:

🎨 7. Trace Inversion Case Studies (5 Key Domains Showcase)

To demonstrate how Trace Inversion reconstructs logical continuity and eliminates negative entropy, the following interactive panels show the contrast between raw compressed "Reasoning Bubbles" and the fully step-by-step reconstructed chain-of-thought (Learnable CoT) under 5 typical scenarios:

📐 Domain 1: Mathematics (Probability Calculation)

🚀 Domain 2: Physics (Kinematics)

💻 Domain 3: Coding (Algorithm Logic)

🧠 Domain 4: Logical Reasoning (Syllogism)

💡 Domain 5: Core Theory (Reasoning Bubble vs. Learnable CoT)

🤝 8. Collaboration & Training Details

This model is a collaborative milestone achieved with hardware engineer Kyle Hessling. You can follow him on X / Twitter: @KyleHessling1 to keep up with the latest hardware infrastructure and distributed training updates. 🙏

⚠️ 9. Known Training & Deployment Issues (IMPORTANT)

While the 27B dense model architecture is relatively stable, certain low-level framework compatibility issues may still surface during large-scale parameter updates and complex long-context training. It is highly recommended to monitor the following technical risk points during secondary fine-tuning and deployment:

[!CAUTION] Local Fine-Tuning & Deployment Warning: If you attempt to run secondary fine-tuning or merge adapter weights locally, please proceed with caution and be prepared to manually patch model definition files or pin dependency versions strictly.

📚 10. Resources & Guides

👉 GitHub Repository: Jackrong-llm-finetuning-guide Access the repository to dive into the codebase and reproduce our results locally or on Google Colab.

🙏 11. Acknowledgements

Special thanks to:

The Qwen team for providing the powerful Qwen3.6 base model.
Unsloth for providing the highly efficient fine-tuning framework.
Open-source datasets and community contributors.
Kyle Hessling for the close collaboration on this project.

📖 12. Citation

bibtex
@misc{jackrong_qwopus36_27b_v2,
  title        = {Qwopus3.6-27B-v2},
  author       = {Jackrong},
  year         = {2026},
  publisher    = {Hugging Face}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider