pocketsflow

penny-1

README

License: apache-2.0

Quick start

bash

ollama run hf.co/pocketsflow/penny-1:Q4_K_M

Penny reads the next-token probabilities of one answer token, so call it in raw mode with the prompt format below, or use the bundled clients (penny.py, penny.mjs, zero dependencies, calibration baked in):

python

from penny import Pennypenny = Penny(model="hf.co/pocketsflow/penny-1:Q4_K_M")   # Ollama on localhost:11434penny.yes_no("From: security@paypa1-support.com\nSubject: Your account is suspended\n\"We noticed unusual activity. Verify your password within 24 hours at http://paypa1-verify.co/login or your account will be closed.\"", "Is this email a phishing attempt?")# {'yes': True, 'confidence': 0.8315}

Prompt format

text

<state>any text or JSON about the situation</state><question type="choice">What should we do with this refund request?A. Refund in fullB. Offer store creditC. Decline politely</question><answer>

The answer is the next token: a letter A–H (yes_no: A = yes, B = no) or a digit 1–5 for score. Take the softmax over the allowed tokens only and divide the logits by the per-type temperature in penny_config.json.

Results

Accuracy on held-out test sets (a score counts as right within one point). "Unseen domains" are task families that were not in training at all. API models were asked the same prompts; local models are read from logprobs.

Table with columns: Model, General, trained domains, General, unseen domains, Money, trained, Money, unseen, Calibration error (unseen)
Model	General, trained domains	General, unseen domains	Money, trained	Money, unseen	Calibration error (unseen)
Pocketsflow Penny 1	80.2%	79.9%	71.5%	66.6%	5.2%
Qwen3 0.6B (base, untrained)	55.8%	61.8%	45.6%	47.5%	25.0%
Llama 3.2 1B	50.2%	45.8%	44.4%	40.7%	–
Llama 3.2 3B	76.0%	78.6%	65.2%	69.5%	–
Llama 3.1 8B	82.7%	82.8%	68.8%	70.1%	–
Llama 3.3 70B	88.2%	87.9%	81.9%	81.3%	–
gpt-oss-20b	89.7%	92.2%	83.5%	83.3%	–

Automatic decisions. With the thresholds in penny_config.json (choice ≥ 0.79, yes_no ≥ 0.72, score ≥ 0.8): general, trained: 68.2% answered automatically at 89.2% accuracy; general, unseen: 64.5% answered automatically at 90.8% accuracy; money, trained: 46.4% answered automatically at 84.9% accuracy; money, unseen: 47.5% answered automatically at 82.4% accuracy.

Cost and speed. Q4_K_M is 397 MB. On a 4 vCPU / 8 GB server ( $48/ m o n t h) P e nn y d oes ab o u t 2.4 d ec i s i o n s a seco n d, ab o u t$ 7.59 per million decisions, against $27 f or L l ama 3.18 B an d$ 63 for Llama 3.3 70B through a hosted API.

Penny is less accurate than 8B and 70B models. It is meant for the cheap, high-volume decisions where a calibrated "I'm not sure" is worth more than a few points of accuracy.

Training

Base: Qwen/Qwen3-0.6B. LoRA rank 16 on the top 14 of 28 layers, merged.
Loss on the single answer token only. Trained on a CPU.
Data: about 10,101 synthetic decisions written to a known answer by open models, kept only when a blind judge model agreed. No customer data.
Calibration: one temperature per question type, fitted on validation.

Limits

English only. Synthetic training data: real inputs will be messier.
Not for credit or lending, insurance, hiring, medical or legal decisions.
One question per call; long states are truncated.
Scores (1–5) cluster near the middle on unfamiliar inputs, and a score counts as right within one point. Prefer choose or yes / no for decisions that matter.
Not a security control: it can be confidently wrong about a dangerous agent command. Keep a hard allow-list in front of anything destructive.
Confidence is calibrated on our test sets, not on your data. Check it on a sample of your own before automating anything.

Files

Table with columns: File, What
File	What
`model.safetensors` + config/tokenizer	Merged model, bf16 (transformers, vLLM, TGI)
`pocketsflow-penny-1-q4_k_m.gguf`	4-bit GGUF (recommended)
`pocketsflow-penny-1-q8_0.gguf`	8-bit GGUF
`Modelfile`	Ollama model file for the Q4_K_M GGUF
`penny.py`, `penny.mjs`	Clients with calibration built in
`penny_config.json`	Per-type temperatures and auto thresholds

Citation

text

@misc{pocketsflow2026penny,  title  = {Pocketsflow Penny 1: a small open decision model},  author = {Pocketsflow Research},  year   = {2026},  url    = {https://www.pocketsflow.com/research/penny}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

pocketsflow

Model Tree

Base

Qwen/Qwen3-0.6B

Quantized

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Quick start

bash

ollama run hf.co/pocketsflow/penny-1:Q4_K_M

python

from penny import Pennypenny = Penny(model="hf.co/pocketsflow/penny-1:Q4_K_M")   # Ollama on localhost:11434penny.yes_no("From: security@paypa1-support.com\nSubject: Your account is suspended\n\"We noticed unusual activity. Verify your password within 24 hours at http://paypa1-verify.co/login or your account will be closed.\"", "Is this email a phishing attempt?")# {'yes': True, 'confidence': 0.8315}

from penny import Pennypenny = Penny(model="hf.co/pocketsflow/penny-1:Q4_K_M")   # Ollama on localhost:11434penny.yes_no("From: security@paypa1-support.com\nSubject: Your account is suspended\n\"We noticed unusual activity. Verify your password within 24 hours at http://paypa1-verify.co/login or your account will be closed.\"", "Is this email a phishing attempt?")# {'yes': True, 'confidence': 0.8315}

Prompt format

text

<state>any text or JSON about the situation</state><question type="choice">What should we do with this refund request?A. Refund in fullB. Offer store creditC. Decline politely</question><answer>

<state>any text or JSON about the situation</state><question type="choice">What should we do with this refund request?A. Refund in fullB. Offer store creditC. Decline politely</question><answer>

Results

Table with columns: Model, General, trained domains, General, unseen domains, Money, trained, Money, unseen, Calibration error (unseen)
Model	General, trained domains	General, unseen domains	Money, trained	Money, unseen	Calibration error (unseen)
Pocketsflow Penny 1	80.2%	79.9%	71.5%	66.6%	5.2%
Qwen3 0.6B (base, untrained)	55.8%	61.8%	45.6%	47.5%	25.0%
Llama 3.2 1B	50.2%	45.8%	44.4%	40.7%	–
Llama 3.2 3B	76.0%	78.6%	65.2%	69.5%	–
Llama 3.1 8B	82.7%	82.8%	68.8%	70.1%	–
Llama 3.3 70B	88.2%	87.9%	81.9%	81.3%	–
gpt-oss-20b	89.7%	92.2%	83.5%	83.3%	–

Penny is less accurate than 8B and 70B models. It is meant for the cheap, high-volume decisions where a calibrated "I'm not sure" is worth more than a few points of accuracy.

Training

Base: Qwen/Qwen3-0.6B. LoRA rank 16 on the top 14 of 28 layers, merged.
Loss on the single answer token only. Trained on a CPU.
Data: about 10,101 synthetic decisions written to a known answer by open models, kept only when a blind judge model agreed. No customer data.
Calibration: one temperature per question type, fitted on validation.

Limits

English only. Synthetic training data: real inputs will be messier.
Not for credit or lending, insurance, hiring, medical or legal decisions.
One question per call; long states are truncated.
Scores (1–5) cluster near the middle on unfamiliar inputs, and a score counts as right within one point. Prefer choose or yes / no for decisions that matter.
Not a security control: it can be confidently wrong about a dangerous agent command. Keep a hard allow-list in front of anything destructive.
Confidence is calibrated on our test sets, not on your data. Check it on a sample of your own before automating anything.

Files

Table with columns: File, What
File	What
`model.safetensors` + config/tokenizer	Merged model, bf16 (transformers, vLLM, TGI)
`pocketsflow-penny-1-q4_k_m.gguf`	4-bit GGUF (recommended)
`pocketsflow-penny-1-q8_0.gguf`	8-bit GGUF
`Modelfile`	Ollama model file for the Q4_K_M GGUF
`penny.py`, `penny.mjs`	Clients with calibration built in
`penny_config.json`	Per-type temperatures and auto thresholds

Citation

text

@misc{pocketsflow2026penny,  title  = {Pocketsflow Penny 1: a small open decision model},  author = {Pocketsflow Research},  year   = {2026},  url    = {https://www.pocketsflow.com/research/penny}}

@misc{pocketsflow2026penny,  title  = {Pocketsflow Penny 1: a small open decision model},  author = {Pocketsflow Research},  year   = {2026},  url    = {https://www.pocketsflow.com/research/penny}}