GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
huihui-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
huihui-ai
Model Tree
huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated
Input Modalities
Output Modalities
Supported Functionality
This is the NVFP4 quantitative version of huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated created using vllm-project/llm-compressor
This is just an attempt at NVFP4 quantization; no further tests have been conducted. If there are any issues, please leave a message.
text
uv self update cd /vllm/vllm-project/mkdir venvcd /vllm/vllm-project/venv uv venv --python 3.12 --seed --managed-python cd /vllm/vllm-project/venvsource .venv/bin/activate cd /vllm/vllm-projectgit clone ssh://git@github.com/vllm-project/vllm cd /vllm/vllm-project/vllmpip install -e .text
uv pip show vllmVersion: 0.18.1rc1.dev115+gd6bb2a9d9.cu130 uv pip show compressed-tensorsVersion: 0.14.0.1 uv pip show transformersVersion: 4.57.6 uv pip show torchVersion: 2.10.0+cu130 uv pip show llmcompressorwarning: Package(s) not found for: llmcompressortext
cd /vllm/models hf download huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 --local-dir ./huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 --token xxxtext
cd /vllm/vllm-project/venvsource .venv/bin/activate export CUDA_VISIBLE_DEVICES=0unset LD_LIBRARY_PATH export VLLM_USE_FLASHINFER_MOE_FP4=0export VLLM_NVFP4_GEMM_BACKEND=cutlass cd /vllm/models vllm serve huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 \ --tensor-parallel-size 1 \ --max-model-len 16384 \ --trust-remote-code \ --skip-mm-profiling \ --limit-mm-per-prompt '{"image": 0, "video": 0}' \ --enforce-eager \ --gpu-memory-utilization 0.78 \ --disable-custom-all-reducetext
curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4", "messages": [ {"role": "user", "content": "hello"} ], "temperature": 0.7, "top_p": 0.9, "max_tokens": 8192, "stream": false }'If you encounter the following issue: ValueError: The Tokenizer class TokenizersBackend does not exist or is not currently imported.
Modify tokenizer_config.json
TokenizersBackend-> Qwen2Tokenizer
"tokenizer_class": "TokenizersBackend",
Change it to
"tokenizer_class": "Qwen2Tokenizer",
Risk of Sensitive or Controversial Outputs: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated outputs.
Not Suitable for All Audiences: Due to limited content filtering, the model’s outputs may be inappropriate for public settings, underage users, or applications requiring high security.
Legal and Ethical Responsibilities: Users must ensure their usage complies with local laws and ethical standards. Generated content may carry legal or ethical risks, and users are solely responsible for any consequences.
Research and Experimental Use: It is recommended to use this model for research, testing, or controlled environments, avoiding direct use in production or public-facing commercial applications.
Monitoring and Review Recommendations: Users are strongly advised to monitor model outputs in real-time and conduct manual reviews when necessary to prevent the dissemination of inappropriate content.
No Default Safety Guarantees: Unlike standard models, this model has not undergone rigorous safety optimization. huihui.ai bears no responsibility for any consequences arising from its use.
text
bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70geuv self updatecd /vllm/vllm-project/mkdir venvcd /vllm/vllm-project/venvuv venv --python 3.12 --seed --managed-pythoncd /vllm/vllm-project/venvsource .venv/bin/activatecd /vllm/vllm-projectgit clone ssh://git@github.com/vllm-project/vllmcd /vllm/vllm-project/vllmpip install -e .uv pip show vllmVersion: 0.18.1rc1.dev115+gd6bb2a9d9.cu130uv pip show compressed-tensorsVersion: 0.14.0.1uv pip show transformersVersion: 4.57.6uv pip show torchVersion: 2.10.0+cu130uv pip show llmcompressorwarning: Package(s) not found for: llmcompressorcd /vllm/modelshf download huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 --local-dir ./huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 --token xxxcd /vllm/vllm-project/venvsource .venv/bin/activateexport CUDA_VISIBLE_DEVICES=0unset LD_LIBRARY_PATHexport VLLM_USE_FLASHINFER_MOE_FP4=0export VLLM_NVFP4_GEMM_BACKEND=cutlasscd /vllm/modelsvllm serve huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4 \ --tensor-parallel-size 1 \ --max-model-len 16384 \ --trust-remote-code \ --skip-mm-profiling \ --limit-mm-per-prompt '{"image": 0, "video": 0}' \ --enforce-eager \ --gpu-memory-utilization 0.78 \ --disable-custom-all-reducecurl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated-NVFP4", "messages": [ {"role": "user", "content": "hello"} ], "temperature": 0.7, "top_p": 0.9, "max_tokens": 8192, "stream": false }'