GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
macrocosm-os
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
macrocosm-os
Model Tree
Qwen/Qwen3.5-4B-Base
Input Modalities
Output Modalities
Supported Functionality
bash
pip install torch transformers accelerate safetensors huggingface_hub pillow torchvisionpython
import sysfrom huggingface_hub import hf_hub_download # the loader ships in this reposys.path.insert(0, hf_hub_download("macrocosm-os/iota-decide-4b", "joint_schema_model.py").rsplit("/", 1)[0])from joint_schema_model import load_release_model, systemone model, processor = load_release_model("macrocosm-os/iota-decide-4b", device="cuda") response = systemone(model, processor, { "model": "iota-decide-4b", "state": { "ticket": "Customer asks for a refund 45 days after purchase.", "policy": "Refunds are allowed within 30 days of purchase.", }, "questions": { "eligible": {"type": "noul", "instructions": "Is the customer eligible for a refund under the policy?"}, "route": {"type": "choice", "instructions": "Which team should handle this?", "criteria": {"refunds": "Refunds and billing team", "tech": "Technical support", "sales": "Sales"}}, "urgency": {"type": "score", "instructions": "How urgent is this ticket?", "criteria": {"1": "Low", "2": "Medium", "3": "High"}}, },})print(response["answers"])Output (abridged):
python
{"eligible": {"type": "noul", "noul": 0.0091}, # 0.9% chance it's allowed: not eligible "route": {"type": "choice", "choice": "refunds", "confidence": 0.8322, ...}, "urgency": {"type": "score", "probabilities": {"0": 0.4548, "1": 0.3952, "2": 0.15}, ...}}Quantized builds are in macrocosm-os/iota-decide-4b-GGUF (Q8_0 4.5 GB, Q4_K_M 2.7 GB). They need llama.cpp 0.7.0 or newer:
bash
llama-server -hf macrocosm-os/iota-decide-4b-GGUF:Q8_0 -b 8192 -ub 8192 --port 9931# then POST to http://127.0.0.1:9931/v1/systemone| Question type | What you get |
|---|---|
noul | probability that the statement is true |
choice | the chosen option, with a probability for every option |
score | the chosen level, with a probability for every level |
Ask several questions about the same state in one request. The model decides them jointly.
All numbers below are our own evaluation runs.
| Evaluation | Records | Accuracy |
|---|---|---|
| JevBench public: overall | 231 | 78.8% (182/231) |
| ↳ Easy / Original / Hard | 48 / 72 / 111 | 100% / 97.2% / 57.7% |
| In-distribution test | 71,393 | 83.6% |
| Domains never seen in training | 83,483 | 74.2% |
| Public classification tasks: test / held-out | 21,784 / 15,866 | 86.8% / 80.2% |
| Hard multi-step reasoning cases: test / held-out | 4,707 / 17,853 | 76.9% / 77.5% |
| Base | Qwen3.5-4B-Base |
| Method | LoRA (rank 16) merged into the backbone, plus a joint schema head trained from scratch |
| Objective | soft-target cross-entropy on teacher probabilities, followed by temperature calibration |
| Data | 541k decision records, 1 epoch, max length 4,096 tokens |
| Compute | 4 independent single-GPU machines (NVIDIA A6000), about 26 hours |
Data. About 90% of the records are decision states we generated ourselves: LLM-written workflows, template
states and business / agent scenarios across many domains. They are labelled with teacher probabilities.
The rest comes from open datasets: Open-Jev, civil_comments, typed-decisions, GoEmotions, prompt-injections,
MNLI, SNLI and BoolQ. All sources and their licences are listed in NOTICE. No benchmark data was used for
training.
iota-decide-4b was trained on disaggregated compute: independent single-GPU machines, with no GPU cluster, kept in sync over the internet. It's built with Macrocosmos's IOTA SDK, which will be released publicly soon.
The weights use the joint-schema layout introduced by Cloudflare/clef, so tooling built for that layout can load this model too.
Apache-2.0. About 8.9k training records come from MNLI, SNLI and BoolQ, which are under CC BY-SA licences.
These and all other third-party sources are credited in NOTICE.
pip install torch transformers accelerate safetensors huggingface_hub pillow torchvisionimport sysfrom huggingface_hub import hf_hub_download # the loader ships in this reposys.path.insert(0, hf_hub_download("macrocosm-os/iota-decide-4b", "joint_schema_model.py").rsplit("/", 1)[0])from joint_schema_model import load_release_model, systemone model, processor = load_release_model("macrocosm-os/iota-decide-4b", device="cuda") response = systemone(model, processor, { "model": "iota-decide-4b", "state": { "ticket": "Customer asks for a refund 45 days after purchase.", "policy": "Refunds are allowed within 30 days of purchase.", }, "questions": { "eligible": {"type": "noul", "instructions": "Is the customer eligible for a refund under the policy?"}, "route": {"type": "choice", "instructions": "Which team should handle this?", "criteria": {"refunds": "Refunds and billing team", "tech": "Technical support", "sales": "Sales"}}, "urgency": {"type": "score", "instructions": "How urgent is this ticket?", "criteria": {"1": "Low", "2": "Medium", "3": "High"}}, },})print(response["answers"]){"eligible": {"type": "noul", "noul": 0.0091}, # 0.9% chance it's allowed: not eligible "route": {"type": "choice", "choice": "refunds", "confidence": 0.8322, ...}, "urgency": {"type": "score", "probabilities": {"0": 0.4548, "1": 0.3952, "2": 0.15}, ...}}llama-server -hf macrocosm-os/iota-decide-4b-GGUF:Q8_0 -b 8192 -ub 8192 --port 9931# then POST to http://127.0.0.1:9931/v1/systemone