GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
eivintobias
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
eivintobias
Model Tree
Qwen/Qwen2.5-Coder-1.5B
Input Modalities
Output Modalities
Supported Functionality
markdown
thinking [reasoning] response<decide>speak|stop</decide><verify>known|unknown</verify> [answer] <stop>
Only [answer] should reach the user.
This model emits the Heartly grammar as ordinary multi-token text (the tags are
not tokenizer special tokens), so some front-ends (e.g. LM Studio) may decode
them mangled. The server.py FastAPI loader on GitHub loads this model and
runs every reply through reply_formatter.py, which canonicalises the tags
and returns only the clean answer.
bash
pip install -r requirements.txt # fastapi + uvicorn + transformers + torchpython server.py --model eivintobias/heartly-qwen-code --port 8000curl -X POST http://127.0.0.1:8000/chat \-H "Content-Type: application/json" \-d '{"prompt":"Write a function that reverses a string"}'
Response: {"model":"eivintobias/heartly-qwen-code","raw":"...<decide>...","reply":"<clean answer>"}.
Quick browser test (no curl): open http://127.0.0.1:8000/ — server.py serves an
HTML chat UI at GET /. The first message lazy-loads the model; code answers render
with real line breaks, and the Heartly grammar is stripped by the reply formatter.
Quick offline test (no server): python chat_smoke.py "Write a function that sorts a list".
📦 Model card source: this file (HF_MODEL_CARD_v3.md). When uploaded to
HuggingFace, copy it to README.md on the hub repo.
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torchtok = AutoTokenizer.from_pretrained("eivintobias/heartly-qwen-code")model = AutoModelForCausalLM.from_pretrained("eivintobias/heartly-qwen-code", torch_dtype=torch.float32, device_map="cpu")model.eval()ids = tok.encode("User: Write a function that reverses a string\nAssistant: ", return_tensors="pt")out = model.generate(**ids, max_new_tokens=256, pad_token_id=tok.eos_token_id, do_sample=False)raw = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False)# Strip the grammar -> clean answer:from reply_formatter import format_replyprint(format_reply(raw))
reply_formatter.py(grammar strip) andserver.pyare bundled in this repo (HF clone = flat layout; GitHub =heartly-qwen-code/). Clone it sofrom reply_formatter import format_replyresolves before the offline example.
| File | Description |
|---|---|
config.json | Qwen2ForCausalLM (28 layers, d=1536) + heartly_stop_token_id=9495 |
generation_config.json | default generate params |
chat_template.jinja | standard Qwen chat template |
tokenizer.json / tokenizer_config.json | Qwen BPE tokenizer |
model.safetensors |
sft_dataset_code_v3.jsonl (5,200 conversational Heartly samples)MIT — built on Qwen2.5-Coder (Apache 2.0).
| v3 fine-tuned weights (full fine-tune, not a LoRA adapter) |
server.py | FastAPI server: lazy-loads the model; serves /, /health, /chat; strips Heartly grammar via reply_formatter |
reply_formatter.py | strips thinking / <decide> / <verify> / <stop> -> clean answer; unescapes code newlines |
chat_smoke.py | offline load + chat smoke test (no server) |
requirements.txt | torch, transformers, fastapi, uvicorn, sentencepiece, datasets, scikit-learn, numpy, accelerate, huggingface_hub |