GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
huihui-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
huihui-ai
Model Tree
huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated
Input Modalities
Output Modalities
Supported Functionality
This is the NVFP4 quantitative version of huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated created using vllm-project/llm-compressor
This is just an attempt at NVFP4 quantization; no further tests have been conducted. If there are any issues, please leave a message.
markdown
uv self updatecd /vllm/vllm-project/mkdir venvcd /vllm/vllm-project/venvuv venv --python 3.12 --seed --managed-pythoncd /vllm/vllm-project/venvsource .venv/bin/activatecd /vllm/vllm-projectgit clone ssh://git@github.com/vllm-project/vllmcd /vllm/vllm-project/vllmpip install -e .
markdown
uv pip show vllmVersion: 0.18.1rc1.dev115+gd6bb2a9d9.cu130uv pip show compressed-tensorsVersion: 0.14.0.1uv pip show transformersVersion: 4.57.6uv pip show torchVersion: 2.10.0+cu130uv pip show llmcompressorwarning: Package(s) not found for: llmcompressor
markdown
cd /vllm/modelshf download huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 --local-dir ./huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 --token xxx
markdown
cd /vllm/vllm-project/venvsource .venv/bin/activateexport CUDA_VISIBLE_DEVICES=0unset LD_LIBRARY_PATHexport VLLM_USE_FLASHINFER_MOE_FP4=0export VLLM_NVFP4_GEMM_BACKEND=cutlasscd /vllm/modelsvllm serve huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 \--tensor-parallel-size 1 \--max-model-len 16384 \--trust-remote-code \--skip-mm-profiling \--limit-mm-per-prompt '{"image": 0, "video": 0}' \--enforce-eager \--gpu-memory-utilization 0.78 \--disable-custom-all-reduce
markdown
curl http://localhost:8000/v1/chat/completions \-H "Content-Type: application/json" \-d '{"model": "huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4","messages": [{"role": "user", "content": "hello"}],"temperature": 0.7,"top_p": 0.9,"max_tokens": 8192,"stream": false}'
If you encounter the following issue: ValueError: The Tokenizer class TokenizersBackend does not exist or is not currently imported.
Modify tokenizer_config.json
TokenizersBackend-> Qwen2Tokenizer
"tokenizer_class": "TokenizersBackend",
Change it to
"tokenizer_class": "Qwen2Tokenizer",
Risk of Sensitive or Controversial Outputs: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated outputs.
Not Suitable for All Audiences: Due to limited content filtering, the model’s outputs may be inappropriate for public settings, underage users, or applications requiring high security.
Legal and Ethical Responsibilities: Users must ensure their usage complies with local laws and ethical standards. Generated content may carry legal or ethical risks, and users are solely responsible for any consequences.
Research and Experimental Use: It is recommended to use this model for research, testing, or controlled environments, avoiding direct use in production or public-facing commercial applications.
Monitoring and Review Recommendations: Users are strongly advised to monitor model outputs in real-time and conduct manual reviews when necessary to prevent the dissemination of inappropriate content.
No Default Safety Guarantees: Unlike standard models, this model has not undergone rigorous safety optimization. huihui.ai bears no responsibility for any consequences arising from its use.
markdown
bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge