GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
bmwlab-ntust
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
bmwlab-ntust
Model Tree
Qwen/QwQ-32B
Input Modalities
Output Modalities
Supported Functionality
bash
python3.12 -m venv .venvbash
source .venv/bin/activatebash
pip install --upgrade pipbash
pip install -r requirements.txtbash
hf auth loginbash
hf download bmwlab-ntust/log_copilot_32b_with_rag --local-dir /home/dgx/johnson/models/log_copilot_32bbash
python3 -m vllm.entrypoints.openai.api_server \ --model /home/dgx/johnson/models/log_copilot_32b \ --max-model-len 4096 \ --dtype bfloat16 \ --port 8000 --gpu-memory-utilization 0.8bash
curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "/home/dgx/johnson/models/log_copilot_32b", "messages": [ {"role": "user", "content": "Here is a snippet of OAI gNB error log: [ERROR] RRC Connection Reestablishment Reject. Which parameter configuration might be misconfigured, and how can I fix the gNB configuration?"} ], "temperature": 0.2, "max_tokens": 1024 }'bash
watch -n 2 nvidia-smihf download bmwlab-ntust/log_copilot_32b_with_rag --local-dir /home/dgx/johnson/models/log_copilot_32bpython3 -m vllm.entrypoints.openai.api_server \ --model /home/dgx/johnson/models/log_copilot_32b \ --max-model-len 4096 \ --dtype bfloat16 \ --port 8000 --gpu-memory-utilization 0.8curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "/home/dgx/johnson/models/log_copilot_32b", "messages": [ {"role": "user", "content": "Here is a snippet of OAI gNB error log: [ERROR] RRC Connection Reestablishment Reject. Which parameter configuration might be misconfigured, and how can I fix the gNB configuration?"} ], "temperature": 0.2, "max_tokens": 1024 }'