GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Alextgt
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Alextgt
Model Tree
Qwen/Qwen3-0.6B
Input Modalities
Output Modalities
Supported Functionality
json
{"entities": [{"title": "...", "type": "CONCEPT", "description": "..."}],"relationships": [{"source": "...", "target": "...", "description": "...", "strength": 8.0}]}
source/target must equal a title in entities. The model returns
{"entities": [], "relationships": []} when nothing is extractable (hard-negative training).
The student prompt lives in the project repo (kg_contract/student_prompt.py); it is derived
from the official GraphRAG extraction prompt and uses one field vocabulary
(title/type/description, source/target/description/strength). At inference the project injects
a schema prefix ({"entities": [) to keep small models structurally compliant.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelrepo = "<this repo id>"base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", torch_dtype=torch.bfloat16, device_map="auto")tok = AutoTokenizer.from_pretrained(repo)model = PeftModel.from_pretrained(base, repo)prompt = STUDENT_PROMPT + "\n\n" + passage # see the repomsgs = [{"role": "user", "content": prompt}]text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=4096, do_sample=False)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Canonical masked recipe (Unsloth, one RTX 4090): Qwen3 in no-think chatml,
response-only loss (train_on_responses_only), non-packed (unsloth#5230),
r=32 / alpha=32, lr 5e-5 cosine, 5 epochs, cutoff 6144, bs 2 × grad-accum 4
(effective 8), bf16, 2,575 training passages.
Referent-level (same real-world entity) on a fixed 200-passage sample; judge = Qwen3 thinking. Reference baseline (same recipe, 0.6B) on this sample: entity R/P/F1 = 0.534 / 0.723 / 0.614.
| metric | value |
|---|---|
| entity micro R / P / F1 | 0.536 / 0.699 / 0.607 |
| schema valid | 0.919 |
| hallucination rate | 0.166 |
Entity recall is monotone in size across the capacity line (0.536 → 0.590 → 0.664 case-tolerant 4B). The 0.6B point matches the reference within the sample's ±2.1 pt CI.
serve/ (GGUF / Gradio) is not implemented.MIT (adapters). Base models: Qwen3 (Apache-2.0). Teacher labels from the official Microsoft GraphRAG prompt (MIT). Training data is not redistributed.