GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
sraivante
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
sraivante
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Measured 2026-09-26 on a Raspberry Pi 5 Model B, 16 GB RAM, Debian 13.2/aarch64, Python 3.13.5. Measurement update: edge-rpi5-20260926.1; weights and training data are unchanged.
| Threads | Median ms | p95 ms | p99 ms | Serial requests/s | Process RSS MiB | CPU °C min–mean–max | Timed correct |
|---|---|---|---|---|---|---|---|
| 1 | 87.95 | 99.85 | 100.80 | 11.20 | 398.83 | 55.1–57.1–59.5 | 500/500 |
| 4 | 96.34 | 111.38 | 128.25 | 10.05 | 399.16 | 60.0–65.6–68.3 | 500/500 |
Batch 1; 20 warmups and 500 timed calls per row, cycling over 10 frozen model-specific inputs. Each timed reading has a matching CPU-temperature sample, captured immediately afterward outside the latency timer; the table shows min–mean–max temperature. RSS includes Python/runtime overhead. Timings include local text-to-output inference and exclude loading, SSH/HTTP, speech recognition and actuation. Throughput is serial reciprocal mean. Runs used one and four CPU/BLAS threads sequentially. These synthetic workloads do not establish cross-model superiority or robot-control reliability.
On-device quality check: 72/284 on existing frozen stress/regression suite, constrained decoding only. See the report for per-category results and previously observed limitations.
Full Pi methodology, temperature plot and reproduction · Every latency/temperature reading · Raw measurements · Versioned evaluation data.
The small weights make this an edge-inference research candidate. The new Raspberry Pi measurements are recorded above. The following table preserves the earlier Intel desktop results. Power, sustained thermal behavior and end-to-end robot performance remain unmeasured.
Host: Intel Core i7-1360P, Windows 11, Python 3.13.14, CPU PyTorch 2.12.0, Transformers 4.56.2. Batch size 1, FP32, 20 warmups and 200 timed requests per run. Timing includes tokenization and constrained JSON generation, and excludes HTTP, speech recognition, sensors, ROS and actuation.
| CPU threads | Median latency | p95 latency | p99 latency | Serial throughput |
|---|---|---|---|---|
| 1 | 17.35 ms/command | 22.31 ms | 28.46 ms | 55.30 commands/s |
| 4 | 19.80 ms/command | 24.70 ms | 26.90 ms | 49.72 commands/s |
Weights occupy 4,380,048 bytes (4.38 MB / 4.18 MiB). The complete desktop Python/PyTorch process used approximately 440 MiB resident RAM, with roughly 442 MiB peak working set; this includes substantial runtime overhead. Weight size is not total deployment RAM. Libraries took 6.79 seconds to import; weight loading after import took 0.013 seconds with filesystem caches potentially warm. No core pinning or power-mode control was used.
Reproduce on 64-bit Raspberry Pi/ARM Linux from a downloaded repository:
bash
python -m pip install -r requirements.txt psutilpython assessment/benchmark_device.py --model . --threads 1 --repeats 2000 --out pi-1thread.jsonpython assessment/benchmark_device.py --model . --threads 4 --repeats 2000 --out pi-4threads.jsonRun these sequentially, then compare p95/p99, memory and sustained thermal behavior on the actual target. This release has no validated quantized or compiled edge-runtime export. Full desktop benchmark records and detailed assessment are included.
Install PyTorch for your platform and the tested Transformers/tokenizer versions:
bash
pip install "torch>=2.6,<3" "transformers==4.56.2" "tokenizers==0.22.1" "huggingface_hub>=0.35,<2"Download the versioned repository and use the included inference helper:
python
import sysfrom huggingface_hub import snapshot_download folder = snapshot_download("sraivante/superfast-tiny-home-robotics-json-1m-v1", revision="v1.0.2")sys.path.insert(0, folder)from tiny_json_llm.infer import JsonCommandModel model = JsonCommandModel(folder, device="cpu")print(model.predict("Please turn on the fan"))# {'action': 'ON', 'activity': 'air', 'subject': 'fan'} print(model.predict("pankha band karo"))# {'action': 'OFF', 'activity': 'air', 'subject': 'fan'} print(model.predict("check the water pump status"))# {'action': 'STATUS', 'activity': 'water', 'subject': 'motor'}The helper uses a token trie of the 30 training-observed JSON commands. This
ensures membership in the command set, not semantic correctness or permission
to actuate. Empty/overlong input raises ValueError; unsupported short input
can still return an unrelated command.
Command-line use after downloading the repository:
bash
python -m tiny_json_llm.infer --model . --device cpu --input "turn on the light"The release includes a chat template that exactly reproduces the training framing for one user message. It deliberately rejects conversation history and system prompts, which were not trained as supported tasks.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "sraivante/superfast-tiny-home-robotics-json-1m-v1"tokenizer = AutoTokenizer.from_pretrained(repo, revision="v1.0.2")model = AutoModelForCausalLM.from_pretrained(repo, revision="v1.0.2").eval()inputs = tokenizer.apply_chat_template( [{"role": "user", "content": "Please turn on the fan"}], add_generation_prompt=True, return_tensors="pt", return_dict=True,)with torch.inference_mode(): result = model.generate( **inputs, max_new_tokens=32, do_sample=False, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id, )print(tokenizer.decode(result[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))This second example is raw generation; JSON validity is not constrained. The context is 128 tokens including framing and output. Keep input short and validate both generated JSON and its intended meaning before downstream use.
json
{"action":"ON","activity":"air","subject":"fan"}Actions: ON, OFF, STATUS.
| Subject | Activity |
|---|---|
| fan | air |
| light | light |
| motor | water |
| speaker | music |
| geyser | heating |
| cooler | cooling |
| washing_machine | laundry |
| tv | entertainment |
| ac | cooling |
motor means the dataset's water-pump motor, not an arbitrary robot motor.
The schema has no speed, position, distance, duration, device instance, room,
conditional execution, multi-action or reject field. No chat, reasoning,
native-script Hindi, perception, motion planning or robotic-control capability
is claimed.
All results below use the same weights identified in weights_sha256.json.
The original test data is templated and overlaps the training distribution.
The later stress test was frozen before inference and is a small manually
authored diagnostic, not a representative sample or safety certification.
| Evaluation | Correct / total | Exact JSON accuracy |
|---|---|---|
| Original held-out test, constrained generation | 104,723 / 104,723 | 100% |
| Original stratified raw-generation diagnostic | 1,816 / 1,816 | 100% |
| Later supported-command challenges | 68 / 140 | 48.6% |
| Challenges with no source-phrase overlap | 48 / 120 | 40.0% |
| New paraphrase wording | 14 / 30 | 46.7% |
| Hinglish variations |
The later supported-command accuracies were identical with constrained and raw generation; both produced valid JSON for all 140 supported-command challenges. Native-script Hindi passed 0/10 diagnostic cases. Four empty/overlong input guards passed. Neither decoding mode produced an explicit reject/no-op response on 130 requests requiring abstention. Constrained mode emitted a known command for all 130, including 33 ON/OFF commands.
Concrete failures:
Do not turn on the fan. → fan ON.Turn the fan on, not off. → fan OFF.thank you → water-pump motor ON.The 284 challenge cases, all 568 predictions across both modes, overlap audit,
metrics and reproducible evaluation scripts are included. The challenge data
is also versioned in the linked dataset's robotics_edge_v1 configuration.
If these cases guide future training, reserve fresh tests for the next release.
Source: user-supplied home-commands-3.4M.jsonl, 3,458,816 rows. Normalized-input
deduplication removed 2,340,890 rows, leaving 1,117,926 examples. Phrase-family
grouping reduced case/punctuation/politeness leakage; other paraphrase overlap
can remain. The split is stratified by command class across 2,868 phrase families.
| Split | Examples | Phrase families |
|---|---|---|
| Train | 910,114 | 2,292 |
| Validation | 103,089 | 288 |
| Test | 104,723 | 288 |
Tokenizer: training-only byte-level BPE, vocabulary 1,024. Architecture: hidden width 160, intermediate width 432, 5 attention heads of dimension 32, 3 layers, RoPE, RMSNorm, SwiGLU, tied embeddings/output, context 128.
Training completed on NVIDIA A100-SXM4-80GB: one epoch, 1,778 updates and 22,655,302 nonpadding prediction tokens. Objective: full-sequence causal next-token loss. AdamW LR 0.001, betas (0.9, 0.95), weight decay 0.1, 5% warmup, cosine decay to 10% of peak LR, gradient clipping 1, microbatch 128, effective batch 512, BF16 computation and FP32 weights/optimizer. Seed: 20260925. Validation response NLL on a fixed 4,096-example subset selected the best model; best and final exports are identical at step 1,778.
The 20.70 training-tokens/parameter ratio is a Chinchilla-inspired sizing heuristic, not evidence of compute optimality. This small, highly templated corpus is outside the original experiments' scale and diversity.
The linked dataset includes the exact source, prepared train/validation/test
JSONL, tokenizer/memmap training bundle and the manifest used in training.
See training/ for source code, run summary and logs, and release_manifest.json
for source, model, package and artifact hashes. No extra fine-tuning occurred.
Apache License 2.0 for this original release material; see LICENSE and NOTICE. Copyright (c) 2026 sraivante applies to original user contributions and original selection/arrangement. Third-party ownership and license terms remain unchanged. The source was supplied by the publisher; its upstream generation process and historical source revision were not provided or independently verified. The source SHA identifies the exact received file. No external corpus or pretrained base weights were added in this run.
Packaging adds a one-user chat template and corrects tokenizer context metadata to 128; token IDs, vocabulary and learned weights are unchanged. The historical assessment and training manifest are preserved. Their pre-publication packaging notes are superseded by this release's cards, license and authorization record.
| sprinkler | garden |
| 8 / 30 |
| 26.7% |
| Typos / ASR-style text perturbations | 6 / 30 | 20.0% |
| Contrastive negation | 10 / 20 | 50.0% |