Why this model exists
The first phone model, heed-tasks-only-qwen3-0.6b (EXP-038), wrote tasks like "Maybe call the bank", because its older labels kept hedges. EXP-049 asked two questions:
- Do today's Sonnet 5.5 labels, written under one frozen prompt, fix that and improve task finding?
- Do 2,697 long messy notes (120 to 199 tokens) help on long notes, and how many epochs does the model need?
Both answers were yes (decisions D109 to D112, D115 to D117). On real notes the word-overlap F1 went from 60.9 to 81.7 on the same key, the largest single improvement in the study.
Raw text, no chat template, no system prompt. There is no bucket list in the prompt. The prompt ends with a newline after Tasks:.
Note: <the note text>Tasks:
The answer is a JSON list of strings followed by EOS. An invented example:
{"tasks":["Call the bank at 5","Buy milk"]}
A note with nothing to do gives {"tasks":[]}. Spoken times stay inside the task text. No bucket, time field or reminder is written. Journal items stay in the target.
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "mohitkrconnect/heed-tasks-only-long-qwen3-0.6b"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16) note = "call the bank at five and buy milk on the way home"prompt = f"Note: {note}\nTasks:\n" # raw prompt, no chat template ids = tok(prompt, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=512, do_sample=False) # greedy, as evaluatedprint(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
Weights are bf16. The duplicated tied lm_head was removed before upload (596,049,920 parameters).
Training
Table with columns: Setting, Value| Setting | Value |
|---|
| Run | EXP-049 q06_long_s0, epoch 3 (of 5 epochs, seed 0) |
| Base | Qwen/Qwen3-0.6B-Base, full fine-tune, 596,049,920 parameters |
| Data | 6,914 notes, one row each (4,217 older notes + 2,697 long notes), no punctuation variants. Validation: 451 notes (371 older + 80 long). The 617 sealed-test notes were never put on the training machine |
| Labels | Claude Sonnet 5.5 under one frozen labelling prompt (EXP-046; EXP-048 for the long notes), temperature 0. Hedges like "Maybe" are no longer kept. No per-note human check |
| Recipe | lr 1e-5, AdamW (weight decay 0), 5% warm-up then cosine, batch 8 × 2 = 16, max length 1,024, loss on answer tokens only, bf16 autocast, gradient checkpointing |
| Steps / time | 2,165 steps (5 epochs), 18.86 minutes, best validation loss 0.470 (step 866), peak 14.1 GB |
| Epoch choice | By word-overlap F1, not loss. Best epochs across the four runs were 3, 2, 3, 2 |
| Hardware / cost | 1 × RTX 4090. About 85 minutes of GPU for the whole experiment (both seeds and the arm without long notes), about 1.3atthequoted0.90 per hour (derived estimate, not an invoice) |

Training notes (6,914):
Table with columns: Source, Notes, What it is| Source | Notes | What it is |
|---|
| Simulated notes | 1,108 | Written by Claude Sonnet 5.5, Gemini and GPT-5.4 in my style |
| HeedSet | 1,263 + 399 | Synthetic, written by Claude Opus; 1,263 labelled earlier by Sonnet 5.5, 399 relabelled in EXP-046 |
| Notes I planned | 781 | Planned by me and generated with ChatGPT |
| Claude brain dumps | 480 | Messy notes written by Claude Opus (220) and Sonnet (260) |
| Curated template notes | 36 | Picked by hand from a template-generated draft |
| Real notes | 140 | Real notes from me and five friends |
| Typed notes | 10 | Typed by me |
| Long synthetic notes | 2,697 | Written by Sonnet 5.5, 70% "scattered" personas; 3,550 writer attempts gave 2,858 in range; no near-copies |
Long notes are 39% of the notes, 62% of the tasks and about 60% of the training tokens. 96% of them open with "okay", "so" or "umm", a known habit of the writer model. English only.
Results
Read this first. These are quick word-overlap F1 scores against Claude Sonnet 5.5's labels, not judged scores. They are not comparable with the blind-judge numbers on the app model card, nor with the older-key numbers of the first phone model (91.2 there is on the old key). The old model re-scored on the new key reads 79.7.
Batched generation, validation set, epoch kept:
Table with columns: Old model (EXP-038, re-scored on the new key), Same recipe, new labels, no long notes (mean of 2), This model (long seed 0, ep3), long seed 1 (ep2), long mean of 2 | Old model (EXP-038, re-scored on the new key) | Same recipe, new labels, no long notes (mean of 2) | This model (long seed 0, ep3) | long seed 1 (ep2) | long mean of 2 |
|---|
| Yardstick (199 notes) | 79.7 | 89.6 | 91.0 | 90.2 | 90.6 |
| Real notes (59) | 60.9 | 77.7 | 80.7 | 82.8 | 81.7 |
| HeedSet (140) | 85.3 | 93.1 | 94.1 | 92.5 | 93.3 |
| Older notes (371) | 76.2 | 85.9 | 86.9 | 86.4 | 86.6 |
| Long notes (80) | not scored | 74.7 | 84.2 | 84.0 | 84.1 |
| All (451) | not scored | 81.6 | 85.8 | 85.5 | 85.6 |
Exact lists on real notes: old 35.6%, new labels 40.7% and 49.2%, with long notes 50.8% and 50.8%. On long notes: new labels 6.2% and 6.2%, with long notes 23.8% and 20.0%.
One-at-a-time scoring on 451 notes, the way a phone runs, gave the same picture (means 90.5 / 81.6 / 84.0 / 85.5 for yardstick / real / HeedSet / older, against 90.6 / 81.7 / 84.1 / 85.6 batched). Answers differed on 26 of 96 notes, F1 84.5 against 84.8.
Honest reading. The old model's 91.2 and 77.9 were on the old key. The same answers on the new key score 79.7 and 60.9, so the gain of +9.9 and +16.8 is partly the new models copying Sonnet's style. Only a judge could separate "better" from "more similar", and this experiment was never judged.
Pre-registered predictions: P49.3 (long notes gain at least 5 points) true, at +9.4. P49.4 (within 1 point on older notes) true, at +0.7. P49.5 (best epoch 3 or later in all runs) false (3, 2, 3, 2).
On the phone
Int8 conversion (litert-torch, about 585 MiB) on 30 validation notes, OnePlus CPH2717 (Dimensity 8300): 0 errors; median 2.1 s on real notes, 4.1 s on long notes and 1.3 s on others; slowest 13 s; 1,152 MB; 35 °C. Same list as the pod on 15 of 30 notes (wording differs between int8 and bf16). F1 on those 30 notes: 81.2 int8 against 83.4 on the pod, too few to measure quantisation. On a Samsung M32 (3.73 GB) the same 30 notes ran with 0 errors, 341 s in total, median 10.1 s. The int8 file is not part of this repository.
What it fed, and why it was superseded
It fed the decision to relabel everything with one Sonnet prompt (D109) and to add long notes (D111, D112). The phone ran it from. The next experiments (EXP-050, EXP-054, EXP-061) added the bucket, trained every note in both written and lowercase forms, and added targeted notes.
Limitations
- Tasks only: no bucket, time field or reminder.
- Quick word-overlap scores against one model's labels; judged score not run, sealed test not run.
- No lowercase or unpunctuated training, so it was not prepared for speech-to-text output.
- Long notes are synthetic and "sound unlike phone notes". The real-note slice is only 59 notes.
- Trained on 150 real notes from me and five friends; memorisation was not tested.
- English only.
Links
Data: the synthetic part of the training, validation and test notes, with all five systems' answers and the judge's verdicts on the public test notes, is in mohitkrconnect/heed-voice-notes-to-tasks. My real notes and my friends' are not published.
License
Apache-2.0, the same as the base model. The training labels and most training notes were produced with Claude, Gemini and GPT models. Check those providers' terms before using this model to build a competing general-purpose model.