GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
bmwlab-ntust
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
bmwlab-ntust
Model Tree
Qwen/QwQ-32B
Input Modalities
Output Modalities
Supported Functionality
bash
python3.12 -m venv .venv
bash
source .venv/bin/activate
bash
pip install --upgrade pip
bash
pip install -r requirements.txt
bash
hf auth login
bash
hf download bmwlab-ntust/log_copilot_32b_with_rag --local-dir /home/dgx/johnson/models/log_copilot_32b
bash
python3 -m vllm.entrypoints.openai.api_server \--model /home/dgx/johnson/models/log_copilot_32b \--max-model-len 4096 \--dtype bfloat16 \--port 8000--gpu-memory-utilization 0.8
bash
curl http://localhost:8000/v1/chat/completions \-H "Content-Type: application/json" \-d '{"model": "/home/dgx/johnson/models/log_copilot_32b","messages": [{"role": "user", "content": "Here is a snippet of OAI gNB error log: [ERROR] RRC Connection Reestablishment Reject. Which parameter configuration might be misconfigured, and how can I fix the gNB configuration?"}],"temperature": 0.2,"max_tokens": 1024}'
bash
watch -n 2 nvidia-smi