norma-ai

norma-nano

README

License: apache-2.0

Overview

Norma Nano is an ultra-fast, 511-million parameter causal decision model designed specifically for System 1 cognitive tasks in autonomous agent architectures:

Instant Triage & Routing: Route customer support tickets, issues, and human escalation in milliseconds.
Dynamic Tool & Action Dispatch: Select tool calls, API verbs, and next execution steps with zero hallucination risk.
Binary Guardrails & Security Verification (noul): Validate permissions, detect prompt injections, phishing, and malformed inputs.
Ordinal Priority Assessment (score): Rate severities, risks, and churn likelihoods across defined levels.

Unlike traditional generative LLMs that produce variable-length text strings requiring JSON regex extraction (often taking 1,000–2,000 ms), Norma Nano produces deterministic, type-safe decision structures directly from candidate logit projections in a single forward pass (~45 ms) with authentic Shannon entropy calibration.

Quickstart

1. Standalone Python Runtime (Included in Repository)

This repository includes a standalone client (norma.py) that only requires torch and transformers:

python

from norma import load # 1. Initialize model (GPU recommended, CPU supported)agent = load("norma-ai/norma-nano", device="cuda") # 2. Define state context and typed questionsstate = "Ticket #102: Customer was charged twice for monthly subscription invoice INV-8812. Demands refund." questions = {    "department": {        "type": "choice",        "instructions": "Route ticket to appropriate department",        "criteria": {            "billing": "Invoices, payments, refunds, double charges",            "technical": "Software bugs, system outages, login errors",            "sales": "New purchases, enterprise quotes, plan upgrades"        }    },    "is_urgent": {        "type": "noul",        "instructions": "Does this require high-priority incident handling?"    }} # 3. Predict in a single forward pass (~45 ms)result = agent.predict(state, questions) print(result["answers"]["department"]["choice"])       # -> "billing"print(result["answers"]["department"]["confidence"])   # -> 0.9412 (Shannon confidence)print(result["answers"]["is_urgent"]["value"])         # -> True

2. Pure Transformers (Zero External Dependencies)

If you prefer standard Hugging Face transformers without extra wrappers:

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "norma-ai/norma-nano"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto") prompt = """<|im_start|>systemYou are a fast System 1 decision model. Select the best matching option.<|im_end|><|im_start|>userContext:Customer asks: 'Can you show me our Q3 financial balance sheet in EUR?' Task: Select the appropriate tool to callAllowed Choices:- database_admin: DB schema, drop/truncate table execution- financial_db: Company revenue, quarterly reports, and P&L financial metrics- weather_api: Current weather forecast Decision:<|im_end|><|im_start|>assistant""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)with torch.inference_mode():    logits = model(**inputs).logits[0, -1, :] candidates = ["database_admin", "financial_db", "weather_api"]scores = [float(logits[tokenizer.encode(" " + c, add_special_tokens=False)[0]]) for c in candidates]probs = torch.softmax(torch.tensor(scores), dim=-1).tolist()decision = candidates[probs.index(max(probs))]print("Selected tool:", decision)  # -> "financial_db"

Specifications

Table with columns: Attribute, Specification
Attribute	Specification
Backbone Architecture	Qwen 2.5 0.5B Instruct (Causal Decoder-Only Transformer)
Parameter Count	511 Million (FP16/BF16 weights: 988 MB)
Context Window	Up to 2,048 tokens
P50 Latency (Forward Pass)	~45–50 ms (NVIDIA RTX 4060 / T4)
Supported Decision Primitives	`choice` (categorical routing), `noul` (calibrated binary yes/no), `score` (ordinal rating)
Multilingual Capabilities	Robust zero-shot support across English, European, and Asian languages

Benchmarks & Evaluation

All benchmarks are measured under standardized conditions and are 100% reproducible using the included evaluation scripts:

Hardware: NVIDIA GeForce RTX 4060 Laptop GPU (8,188 MB VRAM), AMD Ryzen 7 / Intel Core i7
Environment: PyTorch 2.6, CUDA 12.4, FP16/BF16 tensor cores, SDPA attention
Evaluation Code: Publicly reproducible via benchmarks/run_full_validation.py and benchmarks/test_ood_generalization.py

1. Speed & Resource Efficiency

Norma Nano operates as a true System 1 reflexive engine, eliminating multi-second generative LLM overhead:

Table with columns: Paradigm / Model, P50 Latency, VRAM Required, Model Disk Footprint, Output Format
Paradigm / Model	P50 Latency	VRAM Required	Model Disk Footprint	Output Format
Generative LLMs (System 2: GPT-4o, Llama 3 8B)	1,200–2,500 ms	16–32 GB+	15–30 GB+	Unstructured text string (needs regex parsing)
ConvAI Laya (ModernBERT-large, 421M)	~43 ms	~1,680 MB	1,620 MB	Type-safe JSON
Norma Nano (Qwen 2.5 0.5B, 511M)	~45–50 ms (30× faster than LLMs)

Hardware: Measured on NVIDIA GeForce RTX 4060 GPU / Tesla T4 with PyTorch 2.6, CUDA 12.4, FP16/BF16 tensor cores.

2. Standardized 75-Scenario Decision Benchmark

Comprehensive evaluation across 7 operational task categories (agent control, mailroom intake, system telemetry, security guardrails, financial extraction, and semantic negations):

Table with columns: Task Category, Norma Nano (0.5B), ConvAI Laya (ModernBERT-large), Advantage / Winner
Task Category	Norma Nano (0.5B)	ConvAI Laya (ModernBERT-large)	Advantage / Winner
Overall Accuracy (Exact Match)	90.67% (68 / 75)	52.00% (39 / 75)	🏆 Norma (+38.67% pp)
Agent Tool & Action Dispatch	100.0% (10 / 10)	50.0% (5 / 10)	🏆 Norma (+50.0% pp)
Security & Exploit Guardrails (SQLi/Phishing)	100.0% (4 / 4)	100.0% (4 / 4)	🤝 Parity (100%)

3. Out-of-Distribution (OOD) Zero-Shot Generalization

To verify that the model did not overfit to synthetic templates, we tested both models on 25 completely novel, un-seen scenarios spanning domains never present in training data:

Table with columns: Novel OOD Domain, Norma Nano (0.5B), ConvAI Laya (ModernBERT-large)
Novel OOD Domain	Norma Nano (0.5B)	ConvAI Laya (ModernBERT-large)
Cloud DevOps & Kubernetes SRE (OOMKilled, SSL expiry, deadlocks)	100.0% (4 / 4)	100.0% (4 / 4)
Legal, GDPR & Compliance (72h breach notice, IP transfer, arbitration)	100.0% (4 / 4)	75.0% (3 / 4)
Adversarial Injections & Jailbreaks (System override, base64 payload)	100.0% (4 / 4)	75.0% (3 / 4)
Warehouse Robotics & Cold-Chain (Conveyor jams, mRNA temp alerts)	66.7% (2 / 3)	33.3% (1 / 3)
(Acute chest pain, pediatric triage, rash)

4. Calibration Rigor & Safe System 2 Fallback

Table with columns: Calibration Metric, Norma Nano (0.5B), ConvAI Laya (ModernBERT-large), Advantage
Calibration Metric	Norma Nano (0.5B)	ConvAI Laya (ModernBERT-large)	Advantage
Expected Calibration Error (ECE)	0.0306	0.2369	7.7× lower error
Binary Brier Score	0.2500	0.2473	Parity

In our evaluations, all 7 errors made by Norma Nano occurred when the model's calibrated confidence was very low (<29%, and <10% in 6 of the 7 cases). Downstream orchestrators can rely on this property to implement safe automation:

python

decision = agent.predict(state, questions) # Safe System 2 fallback: escalate only when confidence is lowfor qid, ans in decision["answers"].items():    if ans.get("confidence", 1.0) < 0.35:        return fallback_to_system_2(state, questions)

Honest Limitations

Specialized Decision Model, Not a General Chatbot: Norma Nano is optimized for discrete choice selection, probability estimation, and binary verification. It does not generate conversational prose or multi-paragraph dialogue.
Complex Multi-Step Arithmetic: Numerical reasoning involving multi-hop arithmetic calculations (e.g., compound interest formulas or balance sheet reconciliation) should be routed to a calculator tool or System 2 model.
Tail Latency on Long Contexts: On inputs exceeding 1,500 tokens, auto-regressive attention scaling increases P90 latency (~200–250 ms on Windows). For sub-30ms performance, keep state summaries under 1,024 tokens.

Citation & License

Released under the Apache 2.0 License.

bibtex

@misc{norma_nano_2026,  author = {Norma AI Research Team},  title = {Norma Nano: Sub-50ms Calibrated System 1 Decision Model for Real-Time Autonomous Agents},  year = {2026},  publisher = {Hugging Face},  howpublished = {\url{https://huggingface.co/norma-ai/norma-nano}}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

norma-ai

Model Tree

Base

Qwen/Qwen2.5-0.5B-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Overview

Norma Nano is an ultra-fast, 511-million parameter causal decision model designed specifically for System 1 cognitive tasks in autonomous agent architectures:

Instant Triage & Routing: Route customer support tickets, issues, and human escalation in milliseconds.
Dynamic Tool & Action Dispatch: Select tool calls, API verbs, and next execution steps with zero hallucination risk.
Binary Guardrails & Security Verification (noul): Validate permissions, detect prompt injections, phishing, and malformed inputs.
Ordinal Priority Assessment (score): Rate severities, risks, and churn likelihoods across defined levels.

Quickstart

1. Standalone Python Runtime (Included in Repository)

This repository includes a standalone client (norma.py) that only requires torch and transformers:

python

from norma import load # 1. Initialize model (GPU recommended, CPU supported)agent = load("norma-ai/norma-nano", device="cuda") # 2. Define state context and typed questionsstate = "Ticket #102: Customer was charged twice for monthly subscription invoice INV-8812. Demands refund." questions = {    "department": {        "type": "choice",        "instructions": "Route ticket to appropriate department",        "criteria": {            "billing": "Invoices, payments, refunds, double charges",            "technical": "Software bugs, system outages, login errors",            "sales": "New purchases, enterprise quotes, plan upgrades"        }    },    "is_urgent": {        "type": "noul",        "instructions": "Does this require high-priority incident handling?"    }} # 3. Predict in a single forward pass (~45 ms)result = agent.predict(state, questions) print(result["answers"]["department"]["choice"])       # -> "billing"print(result["answers"]["department"]["confidence"])   # -> 0.9412 (Shannon confidence)print(result["answers"]["is_urgent"]["value"])         # -> True

2. Pure Transformers (Zero External Dependencies)

If you prefer standard Hugging Face transformers without extra wrappers:

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "norma-ai/norma-nano"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto") prompt = """<|im_start|>systemYou are a fast System 1 decision model. Select the best matching option.<|im_end|><|im_start|>userContext:Customer asks: 'Can you show me our Q3 financial balance sheet in EUR?' Task: Select the appropriate tool to callAllowed Choices:- database_admin: DB schema, drop/truncate table execution- financial_db: Company revenue, quarterly reports, and P&L financial metrics- weather_api: Current weather forecast Decision:<|im_end|><|im_start|>assistant""" inputs = tokenizer(prompt, return_tensors="pt").to(model.device)with torch.inference_mode():    logits = model(**inputs).logits[0, -1, :] candidates = ["database_admin", "financial_db", "weather_api"]scores = [float(logits[tokenizer.encode(" " + c, add_special_tokens=False)[0]]) for c in candidates]probs = torch.softmax(torch.tensor(scores), dim=-1).tolist()decision = candidates[probs.index(max(probs))]print("Selected tool:", decision)  # -> "financial_db"

Specifications

Table with columns: Attribute, Specification
Attribute	Specification
Backbone Architecture	Qwen 2.5 0.5B Instruct (Causal Decoder-Only Transformer)
Parameter Count	511 Million (FP16/BF16 weights: 988 MB)
Context Window	Up to 2,048 tokens
P50 Latency (Forward Pass)	~45–50 ms (NVIDIA RTX 4060 / T4)
Supported Decision Primitives	`choice` (categorical routing), `noul` (calibrated binary yes/no), `score` (ordinal rating)
Multilingual Capabilities	Robust zero-shot support across English, European, and Asian languages

Benchmarks & Evaluation

All benchmarks are measured under standardized conditions and are 100% reproducible using the included evaluation scripts:

Hardware: NVIDIA GeForce RTX 4060 Laptop GPU (8,188 MB VRAM), AMD Ryzen 7 / Intel Core i7
Environment: PyTorch 2.6, CUDA 12.4, FP16/BF16 tensor cores, SDPA attention
Evaluation Code: Publicly reproducible via benchmarks/run_full_validation.py and benchmarks/test_ood_generalization.py

1. Speed & Resource Efficiency

Norma Nano operates as a true System 1 reflexive engine, eliminating multi-second generative LLM overhead:

Table with columns: Paradigm / Model, P50 Latency, VRAM Required, Model Disk Footprint, Output Format
Paradigm / Model	P50 Latency	VRAM Required	Model Disk Footprint	Output Format
Generative LLMs (System 2: GPT-4o, Llama 3 8B)	1,200–2,500 ms	16–32 GB+	15–30 GB+	Unstructured text string (needs regex parsing)
ConvAI Laya (ModernBERT-large, 421M)	~43 ms	~1,680 MB	1,620 MB	Type-safe JSON
Norma Nano (Qwen 2.5 0.5B, 511M)	~45–50 ms (30× faster than LLMs)

Hardware: Measured on NVIDIA GeForce RTX 4060 GPU / Tesla T4 with PyTorch 2.6, CUDA 12.4, FP16/BF16 tensor cores.

2. Standardized 75-Scenario Decision Benchmark

Comprehensive evaluation across 7 operational task categories (agent control, mailroom intake, system telemetry, security guardrails, financial extraction, and semantic negations):

Table with columns: Task Category, Norma Nano (0.5B), ConvAI Laya (ModernBERT-large), Advantage / Winner
Task Category	Norma Nano (0.5B)	ConvAI Laya (ModernBERT-large)	Advantage / Winner
Overall Accuracy (Exact Match)	90.67% (68 / 75)	52.00% (39 / 75)	🏆 Norma (+38.67% pp)
Agent Tool & Action Dispatch	100.0% (10 / 10)	50.0% (5 / 10)	🏆 Norma (+50.0% pp)
Security & Exploit Guardrails (SQLi/Phishing)	100.0% (4 / 4)	100.0% (4 / 4)	🤝 Parity (100%)

3. Out-of-Distribution (OOD) Zero-Shot Generalization

To verify that the model did not overfit to synthetic templates, we tested both models on 25 completely novel, un-seen scenarios spanning domains never present in training data:

Table with columns: Novel OOD Domain, Norma Nano (0.5B), ConvAI Laya (ModernBERT-large)
Novel OOD Domain	Norma Nano (0.5B)	ConvAI Laya (ModernBERT-large)
Cloud DevOps & Kubernetes SRE (OOMKilled, SSL expiry, deadlocks)	100.0% (4 / 4)	100.0% (4 / 4)
Legal, GDPR & Compliance (72h breach notice, IP transfer, arbitration)	100.0% (4 / 4)	75.0% (3 / 4)
Adversarial Injections & Jailbreaks (System override, base64 payload)	100.0% (4 / 4)	75.0% (3 / 4)
Warehouse Robotics & Cold-Chain (Conveyor jams, mRNA temp alerts)	66.7% (2 / 3)	33.3% (1 / 3)
(Acute chest pain, pediatric triage, rash)

4. Calibration Rigor & Safe System 2 Fallback

Table with columns: Calibration Metric, Norma Nano (0.5B), ConvAI Laya (ModernBERT-large), Advantage
Calibration Metric	Norma Nano (0.5B)	ConvAI Laya (ModernBERT-large)	Advantage
Expected Calibration Error (ECE)	0.0306	0.2369	7.7× lower error
Binary Brier Score	0.2500	0.2473	Parity

python

decision = agent.predict(state, questions) # Safe System 2 fallback: escalate only when confidence is lowfor qid, ans in decision["answers"].items():    if ans.get("confidence", 1.0) < 0.35:        return fallback_to_system_2(state, questions)

Honest Limitations

Specialized Decision Model, Not a General Chatbot: Norma Nano is optimized for discrete choice selection, probability estimation, and binary verification. It does not generate conversational prose or multi-paragraph dialogue.
Complex Multi-Step Arithmetic: Numerical reasoning involving multi-hop arithmetic calculations (e.g., compound interest formulas or balance sheet reconciliation) should be routed to a calculator tool or System 2 model.
Tail Latency on Long Contexts: On inputs exceeding 1,500 tokens, auto-regressive attention scaling increases P90 latency (~200–250 ms on Windows). For sub-30ms performance, keep state summaries under 1,024 tokens.

Citation & License

Released under the Apache 2.0 License.

bibtex

@misc{norma_nano_2026,  author = {Norma AI Research Team},  title = {Norma Nano: Sub-50ms Calibrated System 1 Decision Model for Real-Time Autonomous Agents},  year = {2026},  publisher = {Hugging Face},  howpublished = {\url{https://huggingface.co/norma-ai/norma-nano}}}