GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Susu11
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Susu11
Model Tree
Qwen/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
| Path | Contents |
|---|---|
| Repo root | Final PEFT adapters + tokenizer after SFT |
gguf/ | Optional Q8_0/F16 GGUF after python start.py --gguf --qwen |
Trainer checkpoint-* folders stay on the training PC (socratic_qwen3_v9_model/) and are not uploaded.
<think> blocks).Qwen3-4B-Thinking-2507.4-bit NF4 + LoRA (r=8, alpha=16) via python start.py --qwen → train_qwen.py. Data: Susu11/socraticfinetune.
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen3-4B-Instruct-2507"adapter = "Susu11/new4b"tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)model = PeftModel.from_pretrained(model, adapter)messages = [ {"role": "system", "content": "You are a Socratic Science Tutor for a Grade 10 student. Never give the final answer directly."}, {"role": "user", "content": "Why does ice float?"},]text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(tok.decode(out[0, inputs.input_ids.shape[-1]:], skip_special_tokens=True))Local helper: uv run python infer_qwen.py after adapters exist.
After merge + convert: python start.py --gguf --qwen. Then point llama.cpp or Ollama at gguf/socratic-qwen3-q8_0.gguf on this repo.
Same ScienceQA 256-item slice as Phi-3: python start.py --eval --qwen. Compare eval/scienceqa_acc and eval/scienceqa_sri in W&B project science_socratic_qwen3-4b_instruct (WANDB_PROJECT_QWEN).
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = "Qwen/Qwen3-4B-Instruct-2507"adapter = "Susu11/new4b"tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)model = PeftModel.from_pretrained(model, adapter)messages = [ {"role": "system", "content": "You are a Socratic Science Tutor for a Grade 10 student. Never give the final answer directly."}, {"role": "user", "content": "Why does ice float?"},]text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(tok.decode(out[0, inputs.input_ids.shape[-1]:], skip_special_tokens=True))