GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
kriyanshi
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
kriyanshi
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
The user message is JSON: {"utterance", "scene", "focus_id", "history_len"}. The utterance is lowercase speech-to-text without punctuation. Each scene object has id, type, color, x, y and scale.
The assistant reply is a JSON array of ops, or [] to reject.
json
{"utterance":"put a cat on the house","scene":{"objects":[{"id":"house_1","type":"house","color":"brown","x":0.5,"y":0.7,"scale":1}]},"focus_id":null,"history_len":0}→ [{"op":"add","type":"cat","at":{"relation":"on","ref":"house_1"}}]Use exactly this system prompt; the model was trained with it:
text
Turn one spoken drawing instruction into a JSON array of scene ops. Output only the array.Input: {utterance, scene, focus_id, history_len}.Ops: add{type,color?,at?,scale?} recolor{target,color} move{target,to} scale{target,factor} delete{target} undo clear.Types: cat, tree, house, sun, cloud. Colors: red, orange, yellow, green, blue, purple, pink, brown, black, white, gray.target = id already in the scene. at/to = spot (left, right, center, top, bottom, top-left, top-right, bottom-left, bottom-right) or {"relation": left_of|right_of|above|below|on|near, "ref": scene id or {"added_in_op": N}}.Rules: emit only what was said. big 1.5, small 0.6 on add; bigger 1.3, smaller 0.75, twice 2, half 0.5, "N times" N.Selectors (left, bigger, red, middle, other, next to) pick an id; they are not edits. Pronouns need focus_id unless anchored in the utterance.Ignore fillers, restarts and chatter; keep only the corrected item. Repair obvious speech-to-text mishears.Return [] for unsupported objects, colors or extras, missing, ambiguous or deleted refs, vague descriptions, chatter alone, undo with no history, or undo/clear combined with anything.python
import json, torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("kriyanshi/qwen2.5-1.5b-yap-scene-lora")base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "kriyanshi/qwen2.5-1.5b-yap-scene-lora").merge_and_unload().eval() messages = [{"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": json.dumps(mapper_input)}]ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=512, do_sample=False)ops = json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))Guard the output before executing it. Reject the whole batch if any op uses a type, color, spot or relation outside the schema, or a target or ref that isn't a scene id. The reference implementation is guardOps in the project's eval/mappers/qwen.ts. The guarded scores below assume it.
Exact match means the op array equals the label, with key order ignored. All runs use greedy decoding, one run each. The eval set is kriyanshi/yap-scene-eval.
| benchmark | keyword parser | Gemini 3.5 Flash-Lite (prompted) | this model, raw | this model, guarded |
|---|---|---|---|---|
| v2-hard (79) | 37% | 87% | 95% | 95% |
| v2 correct refusals (24) | 23/24 | 18/24 | 24/24 | 24/24 |
| v1 (50) | 96% | 98% | 94% | 96% |
| v1 correct refusals (8) | 8/8 | 7/8 | 6/8 | 7/8 |
Median latency on an A10G, warm container: about 0.7–0.8 s per request.
In Yap's demo build, the Yapper describes a secret scene (e.g. "a blue cat under a tree") without saying the object words. Each utterance goes to this adapter first, with a 3 s timeout and the schema guard. Gemini is the fallback, and a keyword parser is the last resort. After a refusal, a Gemini "best guess" may add the closest object from a loose description, but it can never edit or remove anything.
A canvas check plays a rule-following Yapper script for each of the 18 game scenes and checks the final canvas for objects, colours, counts, sizes and relations:
| scenes correct | lines answered by this model | median latency | |
|---|---|---|---|
| first scripted pass | 13/18 | 49/49 | 0.76 s |
| the game's hint phrases | 18/18 | 50/50 | 0.78 s |
The first-pass failures are listed under Limitations.
[].[])[])[])[], but it adds a cat to both{"utterance":"put a cat on the house","scene":{"objects":[{"id":"house_1","type":"house","color":"brown","x":0.5,"y":0.7,"scale":1}]},"focus_id":null,"history_len":0}→ [{"op":"add","type":"cat","at":{"relation":"on","ref":"house_1"}}]Turn one spoken drawing instruction into a JSON array of scene ops. Output only the array.Input: {utterance, scene, focus_id, history_len}.Ops: add{type,color?,at?,scale?} recolor{target,color} move{target,to} scale{target,factor} delete{target} undo clear.Types: cat, tree, house, sun, cloud. Colors: red, orange, yellow, green, blue, purple, pink, brown, black, white, gray.target = id already in the scene. at/to = spot (left, right, center, top, bottom, top-left, top-right, bottom-left, bottom-right) or {"relation": left_of|right_of|above|below|on|near, "ref": scene id or {"added_in_op": N}}.Rules: emit only what was said. big 1.5, small 0.6 on add; bigger 1.3, smaller 0.75, twice 2, half 0.5, "N times" N.Selectors (left, bigger, red, middle, other, next to) pick an id; they are not edits. Pronouns need focus_id unless anchored in the utterance.Ignore fillers, restarts and chatter; keep only the corrected item. Repair obvious speech-to-text mishears.Return [] for unsupported objects, colors or extras, missing, ambiguous or deleted refs, vague descriptions, chatter alone, undo with no history, or undo/clear combined with anything.import json, torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("kriyanshi/qwen2.5-1.5b-yap-scene-lora")base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "kriyanshi/qwen2.5-1.5b-yap-scene-lora").merge_and_unload().eval() messages = [{"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": json.dumps(mapper_input)}]ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=512, do_sample=False)ops = json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))