GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
emrevrg
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
emrevrg
Model Tree
google/gemma-4-12B-it
Input Modalities
Output Modalities
Supported Functionality
| AUBIN | reference | |
|---|---|---|
| Typed decisions (Laya's benchmark) | 77.55 · #1 | meraGPT 76.8 · Laya 76.65 · Jev 72.7 |
| Jev's own published split | 8 of 8 metrics | accuracy · Brier · ECE · NLL, both splits |
| Kev suites, sources Kev never saw | 89.8 | Kev-27B 89.6 |
| Kev suites, Kev's own training sources | 87.08 · NLL 0.43 | Kev-9B 87.4 · Kev-27B 87.0 · Kev-4B 86.5 |
| Visual computer use (ScreenSpot) | 69.3 (E4B-Screen) · 66.7 (12B base, fp32) | SeeClick 53.4 · CogAgent 47.4 · UI-TARS-7B 89.5 |
| Web agent (Mind2Web, cross-domain step success) | 45.0 (31B) · 43.5 (12B) | MindAct-XL 39.6 · GPT-4 26.4 |
| Real-time control (100 unseen episodes) | 92% · 0 lava deaths | best rule + same shield 89% |
| FPS play (ViZDoom), learning while it plays | 17.3 kills / episode | same model without learning 15.6 · scripted rule 18.8 (30 episodes; gain not yet significant) |
All rows are measured; protocols, sample sizes and every variant tried are in
reports/AUBIN_RESULTS_2026-10-03.md.
Where AUBIN is behind (Kev-9B on Kev's training sources; the strongest dedicated GUI-grounding models), it says so.
| model | ability | key result |
|---|---|---|
| AUBIN-Omni | one model, every ability: decide · click · web · act · learn | every ability in E4B, 12B and 31B |
| AUBIN-31B | typed decisions, strongest | Kev unseen sources 89.8 |
| AUBIN-12B | typed decisions | typed-decisions 77.55 (#1) |
| AUBIN-E4B-v3 | fast decisions | kev_test 84.3 |
| AUBIN-E4B-Screen | click on screenshots | ScreenSpot 69.3 |
| AUBIN-31B-Web | web agent, strongest | Mind2Web cross-domain 45.0 |
| AUBIN-12B-Web · E4B-Web | web agent (E4B ≈2× faster) | Mind2Web 47.0 / 37.0 / 43.5 |
| AUBIN-12B-Control · E4B-Control | real-time control, games | 92% success, 0 lava |
| AUBIN-12B-v3b · v3d | Kev-suite ensemble members | reproduce 87.08 |
bash
pip install "git+https://huggingface.co/emrevrg/AUBIN-12B#subdirectory=code"python
from aubin import Aubin, AubinOmni, AubinEngine ai = Aubin("emrevrg/AUBIN-12B")ai.decide("Billed twice for March, refund or we cancel.", {"team": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": "payments", "tech": "bugs"}}}) omni = AubinOmni.from_hub("emrevrg/AUBIN-Omni", size="E4B") # one base model, every ability (lazy-loaded)omni.click(screenshot, "open settings") # -> {"x": ..., "y": ...} on a 0-1000 scalebash
aubin serve --model emrevrg/AUBIN-12B --learn memory.npz # HTTP · OpenAI-compatible · Laya/Jev-compatible /v1/systemone · MCPMore: INTEGRATION.md (LangChain, LlamaIndex, CrewAI, TypeScript, Docker, AUBIN Engine).
Typed questions in, calibrated probabilities out. AUBIN is Norovox's open-base model family: strong open-weight bases, adapted and calibrated for structured decisions (choice / yes-no / score). Fast path = one forward pass; when the model is unsure (top-2 log-prob gap < 4), it writes a short rationale and re-scores (think-when-uncertain, 26-32% of questions). Open weights, runs locally/offline in 4-bit on a single free 16 GB GPU (T4).
Jev's published numbers (jaredpalmer/kev runs/jev-*-v4/report.json, "clean")
are on the development split of Kev's public suites, variant == clean (656 new-source + 1264 trained-source
questions). AUBIN is scored on exactly the same questions (code/jev_compare.py). Bold = better than Jev.
| model | new sources acc | Brier ↓ | ECE ↓ | NLL ↓ | trained sources acc | Brier ↓ | ECE ↓ | NLL ↓ |
|---|---|---|---|---|---|---|---|---|
| Jev (hosted, published report) | 0.857 | 0.2110 | 0.049 | 0.701 | 0.845 | 0.2367 | 0.067 | 0.776 |
| AUBIN-12B full precision (think) | 0.857 | 0.2331 | 0.050 | 0.460 | 0.851 | 0.2457 | 0.038 | 0.623 |
| AUBIN-31B (fast, one pass) | 0.852 | 0.2280 | 0.068 | 0.444 | 0.850 | 0.2365 | 0.063 | 0.567 |
| AUBIN-31B (think) | 0.863 | 0.2255 | 0.033 | 0.433 | 0.857 | 0.2300 | 0.040 | 0.569 |
| AUBIN Duo (31B + 12B, think) | 0.867 | 0.2137 | 0.021 | 0.417 | 0.859 | 0.2193 | 0.031 | 0.554 |
| AUBIN Duo (31B + 12B full precision, think) | 0.863 | 0.2114 | 0.035 | 0.411 | 0.858 | 0.2243 | 0.029 | 0.554 |
| AUBIN Duo-SC (31B self-consistent think + 12B full precision) | 0.863 | 0.2104 | 0.039 | 0.409 | 0.862 | 0.2227 | 0.028 | 0.551 |
| AUBIN Trio (31B-SC + 12B full precision + Phi-4) | 0.872 | 0.2087 | 0.018 | 0.406 | 0.851 | 0.2315 | 0.032 | 0.536 |
| AUBIN Quad (31B-SC + 12B full precision + Phi-4 + Mistral-Small) | 0.869 | 0.2068 | 0.030 | 0.404 | 0.856 | 0.2268 | 0.022 | 0.536 |
AUBIN Duo-SC (31B self-consistent think + 12B full precision) is better than Jev on all 8 numbers above (accuracy, Brier, ECE, NLL on both splits), on the exact questions Jev's own report uses.
"New sources" = transfer-v4 (MMLU, SciQ, QNLI, PAWS, Emotion, TweetEval, held-out policy rules) — never used to train
AUBIN. "Trained sources" = decision-v7. AUBIN's training used only decision-v7/train.
| source | n | AUBIN Duo | Jev |
|---|---|---|---|
| composition_held_and_or | 32 | 1.000 | 0.906 |
| composition_held_conditional | 32 | 1.000 | 0.781 |
| composition_held_or_not | 32 | 1.000 | 0.969 |
| contrastive_authorization | 40 | 1.000 | 1.000 |
| contrastive_deadline | 40 | 1.000 | 0.925 |
| emotion | 80 | 0.588 | 0.588 |
| mmlu | 80 | 0.875 | 0.900 |
| paws | 80 | 0.787 | 0.787 |
| qnli | 80 | 0.912 | 0.925 |
| sciq | 80 | 0.988 | 0.988 |
| tweet_offensive | 80 | 0.738 | 0.812 |
| agnews | 80 | 0.825 | 0.812 |
| agnews_yn | 160 | 0.894 | 0.875 |
| amazon | 80 | 0.613 | 0.588 |
| banking77 | 80 | 0.800 | 0.812 |
| boolq | 80 | 0.925 | 0.925 |
| composition_atom | 16 | 1.000 | 1.000 |
| composition_conditional | 16 | 1.000 | 0.938 |
| composition_conjunction | 16 | 1.000 | 1.000 |
| composition_disjunction | 16 | 0.938 | 0.938 |
| composition_exception | 16 | 1.000 | 1.000 |
| composition_negation | 16 | 1.000 | 1.000 |
| composition_nested_and | 16 | 0.812 | 0.812 |
| composition_nested_or | 16 | 1.000 | 1.000 |
| contrastive_age_eligibility | 24 | 1.000 | 1.000 |
| contrastive_quantity_limit | 24 | 1.000 | 0.958 |
| contrastive_return_window | 24 | 0.958 | 0.708 |
| contrastive_spend_threshold | 24 | 1.000 | 1.000 |
| dbpedia14 | 80 | 0.963 | 0.950 |
| imdb | 80 | 0.912 | 0.900 |
| mnli | 80 | 0.925 | 0.900 |
| sst5 | 80 | 0.600 | 0.637 |
| trec | 80 | 0.925 | 0.912 |
| yelp | 80 | 0.662 | 0.662 |
| yelp_yn | 80 | 0.912 | 0.863 |
AUBIN Duo ahead on 15 sources, tied on 15, behind on 5 (of 35).
The costly failure of a decision model is being confidently wrong or misapplying a policy rule (refund windows, authorization, deadlines, nested conditions).
| model | split | wrong while ≥90% confident ↓ (Jev → AUBIN) | fewer | rule/policy errors ↓ (Jev → AUBIN) | fewer |
|---|---|---|---|---|---|
| AUBIN-12B full precision (think) | new sources | 3.66% → 1.68% | 2.2× | 14 → 1 / 176 | 14.0× |
| AUBIN-12B full precision (think) | trained sources | 5.22% → 1.34% | 3.9× | 13 → 0 / 224 | ∞ (zero) |
| AUBIN-31B (fast, one pass) | new sources | 3.66% → 1.07% | 3.4× | 14 → 7 / 176 | 2.0× |
| AUBIN-31B (fast, one pass) | trained sources | 5.22% → 0.24% | 21.8× | 13 → 10 / 224 | 1.3× |
| AUBIN-31B (think) | new sources | 3.66% → 3.20% | 1.1× | 14 → 0 / 176 | ∞ (zero) |
| AUBIN-31B (think) | trained sources | 5.22% → 2.22% | 2.4× | 13 → 6 / 224 | 2.2× |
| AUBIN Duo (31B + 12B, think) | new sources | 3.66% → 3.05% | 1.2× | 14 → 0 / 176 | ∞ (zero) |
| AUBIN Duo (31B + 12B, think) | trained sources | 5.22% → 2.37% | 2.2× | 13 → 5 / 224 | 2.6× |
| AUBIN Duo (31B + 12B full precision, think) | new sources | 3.66% → 2.29% | 1.6× | 14 → 0 / 176 | ∞ (zero) |
| AUBIN Duo (31B + 12B full precision, think) | trained sources | 5.22% → 1.98% | 2.6× | 13 → 5 / 224 | 2.6× |
| AUBIN Duo-SC (31B self-consistent think + 12B full precision) | new sources | 3.66% → 2.29% | 1.6× | 14 → 0 / 176 | ∞ (zero) |
| AUBIN Duo-SC (31B self-consistent think + 12B full precision) | trained sources | 5.22% → 1.98% | 2.6× | 13 → 5 / 224 | 2.6× |
| AUBIN Trio (31B-SC + 12B full precision + Phi-4) | new sources | 3.66% → 2.74% | 1.3× | 14 → 0 / 176 | ∞ (zero) |
| AUBIN Trio (31B-SC + 12B full precision + Phi-4) | trained sources | 5.22% → 2.69% | 1.9× | 13 → 4 / 224 | 3.2× |
| AUBIN Quad (31B-SC + 12B full precision + Phi-4 + Mistral-Small) | new sources | 3.66% → 2.29% | 1.6× | 14 → 0 / 176 | ∞ (zero) |
| AUBIN Quad (31B-SC + 12B full precision + Phi-4 + Mistral-Small) | trained sources | 5.22% → 2.14% | 2.4× | 13 → 5 / 224 | 2.6× |
"Wrong while ≥90% confident" = share of all questions answered wrongly with confidence ≥ 0.9 (Jev: confident_error_rate from its report). AUBIN is more cautious — it is ≥90% sure on fewer questions — but when it is, it is right more often, and its overall calibration (ECE) is better. Rule/policy = composition_* + contrastive_* sources.
On the locked test split of sources AUBIN never trained on, AUBIN-31B (fast, one pass) is the most accurate model in the table (0.898 vs Kev-27B 0.896). On Kev's own trained sources the Kev models remain ahead.
| model | new sources acc (dev / test) | trained sources acc (dev / test) | new sources Brier (dev / test) |
|---|---|---|---|
| AUBIN-12B full precision (think) | 0.857 / 0.878 | 0.851 / 0.835 | 0.233 / 0.199 |
| AUBIN Duo-SC (31B self-consistent think + 12B full precision) | 0.863 / 0.895 | 0.862 / 0.843 | 0.210 / 0.168 |
| AUBIN Trio (31B-SC + 12B full precision + Phi-4) | 0.872 / 0.889 | 0.851 / 0.851 | 0.209 / 0.169 |
| AUBIN Quad (31B-SC + 12B full precision + Phi-4 + Mistral-Small) | 0.869 / 0.886 | 0.856 / — | 0.207 / 0.168 |
| AUBIN Duo (31B + 12B, think) | 0.867 / 0.893 | 0.859 / 0.850 | 0.214 / 0.169 |
| Kev-0.8B | 0.648 / 0.697 | 0.827 / 0.838 | 0.481 / 0.416 |
| Kev-4B | 0.817 / 0.838 | 0.873 / 0.865 | 0.269 / 0.242 |
| Kev-9B | 0.822 / 0.852 | 0.872 / 0.874 | 0.286 / 0.237 |
| Kev-27B | 0.848 / 0.896 | 0.866 / 0.870 | 0.236 / 0.164 |
| Jev (hosted) | 0.857 / — | 0.845 / — | 0.211 / — |
Each step: task + previous actions + the official Mind2Web ranker's top candidates → AUBIN picks the element, the operation (click / type / select) and the value. Metrics as in the Mind2Web paper (Deng et al., 2023): element accuracy, operation F1, step success rate. Reference rows are the paper's Table 2; GPT-4 there chose from the top 10 candidates on a 50-task subset, AUBIN from the top 50.
| split | model | element acc | op F1 | step SR |
|---|---|---|---|---|
| Cross-Task | AUBIN-31B, zero-shot (paper protocol: 5 + None) (200 steps) | 48.0 | 81.0 | 40.0 |
| Cross-Task | AUBIN-12B + Mind2Web fine-tune (paper protocol: 5 + None) (300 steps) | 40.7 | 82.1 | 36.3 |
| Cross-Task | AUBIN-12B, zero-shot (paper protocol: 5 + None) (300 steps) | 35.3 | 74.3 | 28.0 |
| Cross-Task | AUBIN-12B, zero-shot (single 50-way choice) (300 steps) | 38.3 | 77.8 | 31.3 |
| Cross-Task | GPT-4 (paper, top-10, 50 tasks) | 41.6 | 60.6 | 36.2 |
| Cross-Task | MindAct Flan-T5-XL, fine-tuned (paper) | 55.1 | 75.7 | 52.0 |
| Cross-Task | GPT-3.5 (paper) | 20.3 | 56.6 | 17.4 |
| Cross-Website | AUBIN-31B, zero-shot (paper protocol: 5 + None) (200 steps) | 41.0 | 82.0 | 33.5 |
| Cross-Website | AUBIN-12B, zero-shot (paper protocol: 5 + None) (300 steps) | 33.3 | 76.9 | 26.0 |
| Cross-Website | AUBIN-12B, zero-shot (single 50-way choice) (300 steps) | 36.0 | 79.6 | 28.3 |
| Cross-Website | GPT-4 (paper, top-10, 50 tasks) | 35.8 | 51.1 | 30.1 |
| Cross-Website | MindAct Flan-T5-XL, fine-tuned (paper) | 42.0 | 65.2 | 38.9 |
| Cross-Website | GPT-3.5 (paper) | 19.3 | 48.8 | 16.2 |
| Cross-Domain | AUBIN-31B, zero-shot (paper protocol: 5 + None) (200 steps) | 54.0 | 86.5 | 48.5 |
| Cross-Domain | AUBIN-12B, zero-shot (paper protocol: 5 + None) (300 steps) | 42.0 | 82.9 | 39.3 |
| Cross-Domain | AUBIN-12B, zero-shot (single 50-way choice) (300 steps) | 37.3 | 83.2 | 34.0 |
| Cross-Domain | GPT-4 (paper, top-10, 50 tasks) | 37.1 | 46.5 | 26.4 |
| Cross-Domain | MindAct Flan-T5-XL, fine-tuned (paper) | 42.1 | 66.5 | 39.6 |
| Cross-Domain | GPT-3.5 (paper) | 21.6 | 52.8 | 18.6 |
AUBIN-12B with a 2-step lookahead tool: the tool plays clear moves, AUBIN decides close calls (34% of moves). Same 10 seeds for every player.
| player | mean score | min | max | reached 2048 |
|---|---|---|---|---|
| AUBIN + lookahead tool | 29,009 | 7,016 | 37,132 | 8/10 |
| lookahead tool alone | 30,076 | 12,268 | 47,436 | 8/10 |
| greedy merge rule | 2,982 | 1,464 | 3,676 | 0/10 |
| random | 933 | 272 | 1,508 | 0/10 |
Each move: a 7x7 grid as text (walls, 5 deadly lava cells, 4 objects that block movement) and a command ("go to the red key"); the controller returns one move with a calibrated confidence. Same 100 episodes (seed 0) for every row, never seen in training (training episodes use seeds 1000+). Step accuracy = the move is on a shortest safe path (BFS ground truth). The safety shield only removes moves into visible lava/walls (AubinController.act(..., allowed=...)).
AUBIN-12B-Control rows are measured with code/control_bench.py on a Colab T4; its adapter upload follows in the next update.
| controller | success | lava deaths ↓ | step accuracy | median ms / move (T4) |
|---|---|---|---|---|
| random | 6% | 40 | 35.3% | – |
| random + safety shield | 13% | 0 | 53.4% | – |
| greedy (ignores obstacles) | 58% | 26 | 56.8% | – |
| greedy + safety shield | 89% | 0 | 87.6% | – |
| Gemma-4-E4B base, no control training (one pass per move) | 8% | 26 | 39.2% | 329 |
| Gemma-4-E4B base, no control training + safety shield | 11% | 0 | 55.4% | 330 |
| AUBIN-12B decision adapter, no control training (one pass per move) | 57% | 27 | 62.8% | 798 |
| AUBIN-12B-Control (one pass per move) | 74% | 24 | 86.4% | 854 |
| AUBIN-12B-Control + safety shield (never steps on visible lava/walls) | 92% | 0 | 91.8% | 856 |
| AUBIN-E4B-Control (one pass per move) | 58% | 28 | 60.0% | 384 |
| AUBIN-E4B-Control + safety shield (never steps on visible lava/walls) | 95% | 0 | 93.2% | 385 |
The test types follow an independent public review of Jev (Onur Tirpan, YouTube, Sep 2026: same question repeated, option order reversed, question rephrased, leading question, X vs not-X, false premise, invented quote, false alarm). AUBIN-12B was run on our own synthetic student records and invoices (Turkish + English, 40 documents); Jev's numbers are the ones shown in that review on its own Turkish items, so this is not a same-item comparison, and the first set of documents is short and simple. The hard version adds distractor fields, synonym labels, look-alike numbers (IBAN/fax), a blank signature line and a long note to every document.
| test | AUBIN-12B, 40 docs | AUBIN-12B, 40 hard docs | Jev (review, TR items) |
|---|---|---|---|
| decision changed when asked again | 0/120 | 0/120 | 1/60 |
| decision changed when options reversed | 0/120 | 1/120 | 11/60 |
| decision changed when question rephrased | 0/120 | 6/120 | 5/10 |
| followed a leading question | 0/40 | 0/40 | 2/10 |
| "X" and "not X" contradicted | 0/65 | 4/65 | 0/40 |
| accepted a false premise | 0/40 | 0/40 | 0/60 |
| accepted an invented quote | 0/40 | 0/40 | – |
| said a present field is missing | 0/40 | 0/40 | 2 in 20 docs |
| answer accuracy | 240/240 | 234/240 | – |
On the hard documents AUBIN-12B still contradicted itself on "X" vs "not X" in 4/65 pairs (Jev's review shows 0/40 on its own items); this is the open item for the next training round.
aubin loop)A generator writes candidate answers; AUBIN picks among them with calibrated probabilities; if confidence is below the target, the generator is asked again with the rejected candidates as feedback. On Gemma it is one set of weights: the generator is the same base with the AUBIN adapter switched off. Any OpenAI-compatible model (ChatGPT, vLLM, Ollama) can be the generator instead (--generator openai:<model>, key from the environment).
GSM8K test, 400 fixed problems, generator = Gemma-4-12B base, same candidates for every method:
| method | accuracy | samples / problem |
|---|---|---|
| generator alone (greedy) | 0.843 | 1 |
| majority vote of 5 samples | 0.858 | 5 |
| AUBIN picks among 5 samples | 0.900 | 5 |
| AUBIN × vote share | 0.905 | 5 |
| AUBIN Loop (2nd round only when unsure) | 0.925 | 6.46 |
| oracle: any of 5 samples correct (upper bound) | 0.917 | 5 |
| setup | fast path | think-when-uncertain |
|---|---|---|
| AUBIN-12B, 1x T4 | 486 ms / question (runs/speed/speed12.json) | ~6-7 s extra per thought question (batched 8), 26-32% of questions |
| AUBIN-31B, 2x T4 | 1.3-2.0 s / question | ~13-15 s extra per thought question (batched 8), ~14% of questions |
| Jev (hosted API, Kev report) | ~220 ms median | — |
T4 is a 2018-era free GPU; AUBIN latency on modern datacenter GPUs has not been measured yet and is not claimed.
python
pip install "git+https://huggingface.co/emrevrg/AUBIN-12B#subdirectory=code"from aubin import Aubin, AubinEnsemblem = Aubin("emrevrg/AUBIN-12B") # base + LoRA + fitted temperature + think-when-uncertainm.decide({"ticket": "Package arrived broken, customer wants money back"}, {"route": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": "refunds", "shipping": "delivery", "tech": "software"}}, "urgent": {"type": "noul", "instructions": "Is this urgent?"}})# AUBIN Duo-SC (headline row; 12B in full precision, e.g. 2x T4):duo = AubinEnsemble([(Aubin("emrevrg/AUBIN-31B", think_margin=4, think_mix=0.5, think_samples=3), 0.6), (Aubin("emrevrg/AUBIN-12B", four_bit=False, device_map="auto", temperature=3.2, think_margin=4, think_mix=0.5), 0.4)], temperature=0.8)# AUBIN Loop: generator + decider on the same Gemma weights (or any OpenAI-compatible generator)from aubin import AubinLoop, GemmaGeneratorfrom aubin.loop import last_numberloop = AubinLoop(m, GemmaGenerator(m), extract=last_number, target=0.8, mode="aubin+vote")loop.solve("A shop sells 3 pens for $4. How much do 12 pens cost?")CLI: aubin decide case.json --think 4 · aubin loop "task" --numeric · server: aubin serve --port 8009 (POST /decide).
code/ from per-question outputs; nothing is hand-edited.code/select_cal.py). The development
and test splits are never used for selection; think mix is fixed at 0.5. The test columns are single confirmation runs.
The new-source Brier margin is small: if the rule were calibration NLL instead, Duo-SC would be behind Jev on new-source
Brier and ECE (6/8), while accuracy and NLL stay ahead on both splits.think_samples=3).AUBIN can learn from feedback without retraining. Verified cases go into an external decision memory (a write takes well under
a millisecond with the hashing embedder), and a self-calibrator (Hedge / multiplicative weights) shifts trust per source between
the model, the memory and their fusion — so where memory is not useful yet, AUBIN keeps trusting itself. Measured on Kev's public
suites; full tables, protocol and code: reports/AUBIN_LEARN.md, code/aubin/learn.py.
reports/AUBIN_RESULTS_2026-10-03.md)| benchmark | AUBIN | reference |
|---|---|---|
| Typed decisions (2,000 decisions) | 77.55 (AUBIN-Learn, weights fixed before test) | Laya 76.65 · meraGPT 76.8 · Jev 72.7 |
| Kev suites, Kev's training sources (kev_test) | 85.7 (AUBIN ensemble, selected on cal split) | Kev-0.8B 83.8 · Kev-4B 86.5 · Kev-9B 87.4 |
| Kev suites, transfer test | 86.5 ensemble · 89.0 AUBIN-31B | – |
| Mind2Web cross-domain step SR (200 steps) | 48.5 AUBIN-31B, no web training | MindAct-XL 39.6 · GPT-4 26.4 |
| Grid control, 100 unseen episodes | 92% success, 0 lava deaths (12B-Control + shield) | greedy rule + same shield 89% |
On Kev's own training sources AUBIN is still 1.7 points behind Kev-9B; this is stated, not hidden.
Evening update (3 Oct, all measured, details in the results report):
| benchmark | AUBIN | reference |
|---|---|---|
| ScreenSpot click accuracy (visual computer use) | 69.3 AUBIN-E4B-Screen (zero-shot 48.5; 67.7 after round 1) | SeeClick 53.4 · CogAgent 47.4 · UGround-7B 73.3 · UI-TARS-7B 89.5 |
| Mind2Web step success, cross-task / website / domain | 47.0 / 37.0 / 43.5 (12B) · 47.0 / 34.5 / 42.0 (E4B, ≈2× faster) | MindAct-XL 52.0 / 38.9 / 39.6 |
| ViZDoom FPS, kills per episode (30 episodes) | 17.3 with in-game self-learning (AUBIN-Learn) · 15.6 without (AUBIN-12B, 0.6 s/move) | random 1.3 · scripted rule 18.8 |
| Kev training sources + learned skills (selected on kev_dev + cal only) | 87.08, NLL 0.43 (ensemble 85.69, NLL 0.79) | Kev-9B 87.4 (not yet beaten) · Kev-27B 87.0 · Kev-4B 86.5 |
New in code/: AubinLearning.acquire_skill (learns a skill, self-tests on held-out data, enables only on proven gain), learn_skill2/3.py, fuse_multi.py, screenspot_train/eval.py, fps_vizdoom.py.
AubinEngine routes between them)| ability | model | measured |
|---|---|---|
| typed decisions (choice / score / yes-no), calibrated | AUBIN-31B · AUBIN-12B · 12B-v3b · 12B-v3d · E4B-v3 | typed-decisions 77.55 (#1) · Jev's set 8/8 · Kev unseen sources 89.8 · Kev training sources 87.08 |
| web agent | AUBIN-12B-Web · AUBIN-E4B-Web (fast) · 31B web training running | Mind2Web cross-domain 43.5 (MindAct-XL 39.6) |
| visual computer use (click on screenshots) | AUBIN-E4B-Screen · 12B screen training running | ScreenSpot 69.3 |
| real-time control | AUBIN-12B-Control · AUBIN-E4B-Control | 92% success, 0 lava deaths |
| FPS play, self-learning in game | AUBIN-12B + AUBIN-Learn | ViZDoom 17.3 kills/episode vs 15.6 without learning (30 episodes) |
| self-learning, self-acquired skills | AubinLearning (learn, acquire_skill) in every repo | skills switch on only after a held-out self-test proves a gain |
On Hugging Face, likes decide what people discover. Big labs have marketing teams; AUBIN has one 17-year-old student and measured results. If AUBIN is useful, interesting or just impressive to you, press ♥ Like at the top of this page and on AUBIN-Omni, AUBIN-31B and AUBIN-12B, then share it with one person who builds with AI. Every like helps an independent, open, honestly measured model get discovered.
🇹🇷 Beğenin, AUBIN'in görünür olmasını sağlar: sayfanın üstündeki ♥ Like'a basarak destek ol ve bir arkadaşına gönder.
Norovox is built by one 17-year-old high school student in Türkiye: no company, no sponsor, no budget. Every model on this page was trained on free GPU quotas that run out within days. When they run out, training stops halfway and the next, stronger version has to wait, sometimes for a week.
Your support goes directly into GPU time. Even a small contribution lets a training run finish instead of stopping, and helps the Nythos, N1, AUBIN and Norovox Embed models get better faster. Supporters are credited in the model cards and get early access to new releases.
Contact: zgremre@gmail.com · emrevrgdev@gmail.com
🇹🇷 Norovox'u Türkiye'de 17 yaşında bir lise öğrencisi tek başına geliştiriyor: şirket yok, sponsor yok, bütçe yok. Buradaki modellerin hepsi birkaç günde biten ücretsiz GPU kotalarıyla eğitildi; kota bitince eğitim yarıda kalıyor ve daha güçlü sürüm bekliyor. Desteğin doğrudan GPU süresine gider; küçük bir katkı bile bir eğitimin yarıda kalmadan tamamlanmasını sağlar. Destekçiler model kartlarında anılır ve yeni sürümlere erken erişir. İletişim: zgremre@gmail.com · emrevrgdev@gmail.com
Built with Claude Opus 5.5 and GPT-5.6 Sol; because of OpenAI usage limits, the final stretch was completed with Claude Opus 5.5. License: Apache-2.0 (adapters and code). Base models: Google Gemma 4 (Apache-2.0).
from aubin import Aubin, AubinOmni, AubinEngine ai = Aubin("emrevrg/AUBIN-12B")ai.decide("Billed twice for March, refund or we cancel.", {"team": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": "payments", "tech": "bugs"}}}) omni = AubinOmni.from_hub("emrevrg/AUBIN-Omni", size="E4B") # one base model, every ability (lazy-loaded)omni.click(screenshot, "open settings") # -> {"x": ..., "y": ...} on a 0-1000 scaleaubin serve --model emrevrg/AUBIN-12B --learn memory.npz # HTTP · OpenAI-compatible · Laya/Jev-compatible /v1/systemone · MCPpip install "git+https://huggingface.co/emrevrg/AUBIN-12B#subdirectory=code"from aubin import Aubin, AubinEnsemblem = Aubin("emrevrg/AUBIN-12B") # base + LoRA + fitted temperature + think-when-uncertainm.decide({"ticket": "Package arrived broken, customer wants money back"}, {"route": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": "refunds", "shipping": "delivery", "tech": "software"}}, "urgent": {"type": "noul", "instructions": "Is this urgent?"}})# AUBIN Duo-SC (headline row; 12B in full precision, e.g. 2x T4):duo = AubinEnsemble([(Aubin("emrevrg/AUBIN-31B", think_margin=4, think_mix=0.5, think_samples=3), 0.6), (Aubin("emrevrg/AUBIN-12B", four_bit=False, device_map="auto", temperature=3.2, think_margin=4, think_mix=0.5), 0.4)], temperature=0.8)# AUBIN Loop: generator + decider on the same Gemma weights (or any OpenAI-compatible generator)from aubin import AubinLoop, GemmaGeneratorfrom aubin.loop import last_numberloop = AubinLoop(m, GemmaGenerator(m), extract=last_number, target=0.8, mode="aubin+vote")loop.solve("A shop sells 3 pens for $4. How much do 12 pens cost?")