GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
vtava
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
vtava
Model Tree
HuggingFaceTB/SmolLM2-135M
Input Modalities
Output Modalities
Supported Functionality
[3, 7, 11, 15, 19, 23, 27]) remain fully dense to preserve global representation stability.mix =0.25−0.50mix =0.10Evaluated on NVIDIA L4 GPU (torch.bfloat16). FastEval uses 50 deterministic zero-shot multiple-choice examples per dataset.
| Benchmark | Task Type | SmolLM2-135M (Base) | FlyFFN-v2 | Δ Difference |
|---|---|---|---|---|
| MMLU-Pro | Multi-domain Reasoning | 8.0% (4/50) | 10.0% (5/50) | +2.0% |
| PIQA | Physical Commonsense | 42.0% (21/50) | 50.0% (25/50) | +8.0% |
| MMMLU-DE | Multilingual (German) | 24.0% (12/50) | 32.0% (16/50) | +8.0% |
| GPQA-Diamond | High-level Science | Gated | Gated | — |
| Macro Average | Overall | 24.7% | 30.7% | +6.0% |
Note on Throughput: While sparse FFN routing improves zero-shot accuracy across small-batch evaluations, token generation speed varies by sequence length due to custom routing overheads on standard CUDA kernels (e.g., ~98–110 q/s on PIQA vs. 173 q/s for standard dense).
| Metric | SmolLM2-135M (Base) | FlyFFN-v2 |
|---|---|---|
| Cross-Entropy Loss (CE) | 2.7227 | 2.9609 |
| Perplexity (PPL) | 15.22 | 19.32 |
Ensure you have the custom layer definition script smollm2_flyffn_v2.py and checkpoint weights biological_flyffn_v2.pt in your working directory.
python
import torchfrom smollm2_flyffn_v2 import SmolLM2FlyFFNv2ForCausalLMfrom transformers import AutoTokenizer model_id = "HuggingFaceTB/SmolLM2-135M"tokenizer = AutoTokenizer.from_pretrained(model_id) # Load model architecture with custom FlyFFN layersmodel = SmolLM2FlyFFNv2ForCausalLM.from_pretrained( "./", torch_dtype=torch.bfloat16, device_map="auto") input_text = "The physical principle behind gravity is"inputs = tokenizer(input_text, return_tensors="pt").to("cuda") outputs = model.generate(**inputs, max_new_tokens=50)print(tokenizer.decode(outputs[0], skip_special_tokens=True))vtavakkoli/TinyCeNN-LMHuggingFaceTB/SmolLM2-135Mfast_eval_50.csv, fast_eval_50.jsonbiological_flyffn_v2.ptimport torchfrom smollm2_flyffn_v2 import SmolLM2FlyFFNv2ForCausalLMfrom transformers import AutoTokenizer model_id = "HuggingFaceTB/SmolLM2-135M"tokenizer = AutoTokenizer.from_pretrained(model_id) # Load model architecture with custom FlyFFN layersmodel = SmolLM2FlyFFNv2ForCausalLM.from_pretrained( "./", torch_dtype=torch.bfloat16, device_map="auto") input_text = "The physical principle behind gravity is"inputs = tokenizer(input_text, return_tensors="pt").to("cuda") outputs = model.generate(**inputs, max_new_tokens=50)print(tokenizer.decode(outputs[0], skip_special_tokens=True))