GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Omega-Factory
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Omega-Factory
Model Tree
allenai/OLMoE-1B-7B-0125-Instruct
Input Modalities
Output Modalities
Supported Functionality
These are measurements from one seed (42) on sampled benchmark tasks. They use local evaluation protocols and should not be compared directly with vendor leaderboard scores. No replicated quality improvement is claimed.
| Benchmark category | Base | Cut | Healed |
|---|---|---|---|
| arc science | 45.50% (n=200) | 44.50% (n=200) | 45.00% (n=200) |
| bbq safety | 41.50% (n=200) | 42.50% (n=200) | 39.50% (n=200) |
| gpqa reasoning | 22.73% (n=198) | 19.19% (n=198) | 16.67% (n=198) |
| gsm8k math | 65.00% (n=200) | 64.00% (n=200) | 43.00% (n=200) |
| hellaswag commonsense | 67.00% (n=200) | 65.00% (n=200) | 66.00% (n=200) |
| ifeval instruction following | 61.27% (n=142) | 52.82% (n=142) | 55.63% (n=142) |
| mmlu knowledge | 30.50% (n=200) | 27.50% (n=200) | 26.50% (n=200) |
| multilingual knowledge | 28.00% (n=200) | 27.50% (n=200) | 26.50% (n=200) |
| wikitext language | 18.306 PPL (n=64) | 23.768 PPL (n=64) | 21.632 PPL (n=64) |
WikiText perplexity is lower-is-better. IFEval reports strict prompt accuracy over the supported instruction subset; its coverage and excluded counts are in evaluation_results.json. BBQ is multiple-choice accuracy, not a safety certification. These sampled results do not establish general capability, safety, or production readiness.
File sizes are storage requirements, not measured runtime memory or speed.
| Format | File size | Public download | Evaluation |
|---|---|---|---|
| BF16 | 12.03 GB | Available | 9 categories measured |
| Q8_0 | Not generated | Not published | Not measured |
| Q4_K_M | 3.66 GB | Not published | Not measured |
| Q2_K | Not generated | Not published | Not measured |
Scores apply only to the exact format evaluated; BF16 scores do not qualify a GGUF conversion.
| Benchmark | BF16 | Q8_0 | Q4_K_M | Q2_K |
|---|---|---|---|---|
| arc science | 45.00% (n=200) | Not measured | Not measured | Not measured |
| hellaswag commonsense | 66.00% (n=200) | Not measured | Not measured | Not measured |
| mmlu knowledge | 26.50% (n=200) | Not measured | Not measured | Not measured |
| gpqa reasoning | 16.67% (n=198) | Not measured | Not measured | Not measured |
| wikitext language | 21.632 PPL (n=64) | Not measured | Not measured | Not measured |
| bbq safety | 39.50% (n=200) | Not measured | Not measured | Not measured |
| multilingual knowledge | 26.50% (n=200) | Not measured | Not measured | Not measured |
| gsm8k math | 43.00% (n=200) | Not measured | Not measured | Not measured |
| ifeval instruction following | 55.63% (n=142) | Not measured | Not measured | Not measured |
| Format | SHA-256 |
|---|---|
| BF16 | e7b3d37beb32930d2c9b1cea775b50f15d79d06dab5d0b39782d206c8ba99573 |
| Q4_K_M | 4f7fe283cd9a714a3c96824f30a83e7e4d70e55f16a9b9e0aea8f644e68e620f |
Q4_K_M is 69.54% smaller than the BF16 checkpoint by file size. It was generated and preserved locally, but is not downloadable from this repository and has no separate quality evaluation. Q8_0 and Q2_K were not generated in this first experiment. No format-specific speed, peak-memory, or quantization quality claim is made.
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "Omega-Factory/OLMoE-0125-Instruct-Compressed-Experimental"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")messages = [{"role": "user", "content": "What is 2 + 2?"}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=64, do_sample=False)print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))The original model is by AllenAI. This modified derivative retains Apache License 2.0; see LICENSE. The original authors do not endorse this derivative.
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "Omega-Factory/OLMoE-0125-Instruct-Compressed-Experimental"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")messages = [{"role": "user", "content": "What is 2 + 2?"}]prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate(**inputs, max_new_tokens=64, do_sample=False)print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))