Compression for the Model

Qwen3.5-35B-A3B-abliterated-v2-MAX

Key Highlights

• Latest Transformers Compatibility Re-sharded and optimized for improved compatibility with recent Transformers releases.

• Optimized Model Sharding Updated shard structure for better storage handling, download reliability, and inference efficiency.

• Stable Inference Pipeline Improved packaging for consistent loading and generation behavior across environments.

• 35B MoE Architecture (A3B) Built on Qwen3.5-35B-A3B, leveraging Mixture-of-Experts design for scalable reasoning capacity.

• Improved Deployment Stability Designed for smoother inference across different hardware configurations and runtimes.

• Preserved Model Behavior No changes to weights or architecture; behavior remains consistent with the base model lineage.

---

Base Model Signatures:

This model has been re-sharded and optimized for the latest Transformers version from the base model: https://huggingface.co/huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated.

---

Quick Start with Transformers

bash
Copy code
pip install transformers==5.5.0
# or
pip install git+https://github.com/huggingface/transformers.git

python
Copy code
from transformers import Qwen3_5MoeForConditionalGeneration, AutoProcessor
import torch

model = Qwen3_5MoeForConditionalGeneration.from_pretrained(
    "prithivMLmods/Qwen3.5-35B-A3B-abliterated-v2-MAX",
    torch_dtype="auto",
    device_map="auto"
)

processor = AutoProcessor.from_pretrained(
    "prithivMLmods/Qwen3.5-35B-A3B-abliterated-v2-MAX"
)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Explain how transformer models work in simple terms."}
        ],
    }
]

text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

inputs = processor(
    text=[text],
    padding=True,
    return_tensors="pt"
).to("cuda")

generated_ids = model.generate(**inputs, max_new_tokens=256)

generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]

output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)

print(output_text)

Intended Use

• Multimodal and Language Research Studying large-scale MoE behavior and inference characteristics.

• Red-Teaming & Evaluation Testing robustness across complex and adversarial prompts.

• High-Performance Deployment Running large MoE models on optimized multi-GPU setups.

• Research Prototyping Experimentation with scalable transformer architectures and deployment workflows.

Limitations & Risks

Important Note: This model inherits the behavior and limitations of its base model.

• Output Variability Responses may vary depending on sampling settings and prompt structure.

• Resource Requirements A 35B MoE model requires significant GPU memory or optimized inference strategies such as quantization or tensor parallelism.

• Deployment Constraints Performance depends heavily on hardware configuration and runtime optimization.

• General Model Limitations May produce incorrect, incomplete, or inconsistent outputs in complex scenarios.