Label
Animal marks a word or phrase that names an animal: a whole animal of any
kind (also specific young-animal words like veulen, kalf), a group of
animals (vee, Dieren), eggs or larvae treated as animals, an animal word
used figuratively (also for a person: wat ben jij een ezel), or a dead
animal written about as an animal. Not marked: people and human body
parts, the general jongen, parts of animals, animal products and food
(herring sold or eaten), and compounds mainly about something else
(Haringh-vanghst). Spans are the animal noun, with a modifier only when it
says which animal is meant (woldraghende vee).
The full guidelines (prompt v3) are in task.json. The adapter was trained
with these instructions as the system prompt, so pass the same prompt at
inference.
Training
- Data: 2,250 sentences, 1,211 of them with an animal, labelled zero-shot
by
gpt-6-luna (reasoning effort medium) under the v3 guidelines. They
were chosen by an active-learning loop with a GysBERT student from a pool
of 940,472 DBNL sentences: half found by a yes/no animal filter, the rest
the sentences GysBERT was least sure about, plus 125 random ones. Every
sentence of the books the dev and test sets come from was removed from the
pool, as were near-copies of the evaluation sentences.
- LoRA: r 16, α 32, dropout 0.05, all linear layers.
- Optimisation: lr 2e-4, 2 epochs, effective batch 16 (2 × 8 gradient
accumulation), keeping the checkpoint with the lowest loss on the dev set.
The dev set's labels are used only for that choice, never for training.
- Precision: bf16 base weights.
- Max length: 1280 tokens (the instructions alone are about 600 tokens).
- Cost: about $1.35 of API calls for the labels, about 1.5 hours on one
L40S.
Evaluation
Scores are token-level and positives-only, plus span-level F1, computed with
genannote after reloading the saved adapter.
Test set: the animals_ner_dutch test set, 250 sentences from 7 DBNL
texts, gold-standard annotations.
Table with columns: Model, token F1 (P / R), span F1| Model | token F1 (P / R) | span F1 |
|---|
| this adapter | 0.890 (0.96 / 0.83) | 0.881 |
| gysbert-animals-nov26-v2 (same labels) | 0.801 (0.89 / 0.73) | 0.763 |
gpt-6-luna, zero-shot (the teacher) | 0.961 (0.96 / 0.97) | 0.953 |
The v3 guidelines were written from the teacher's mistakes on this test set,
which flatters the teacher's score here.
Check set: the 1,192 human-checked sentences of the animals_ner_dutch
training set, which this adapter never trained on (some come from the same
DBNL books as its training sentences).
Table with columns: Model, token F1 (P / R), span F1| Model | token F1 (P / R) | span F1 |
|---|
| this adapter | 0.872 (0.90 / 0.85) | 0.873 |
| gysbert-animals-nov26-v2 (same labels) | 0.780 (0.84 / 0.73) | 0.782 |
gpt-6-luna, zero-shot (the teacher) | 0.902 (0.92 / 0.89) | 0.897 |
metrics.json has the full scores for both sets.
Limitations
- Hardware: the base weights in bf16 need a GPU with 24 GB or more; a
9B model is far slower than an encoder (about 0.4 s per sentence on an
L40S with batched generation).
- Spans come back as text, not character offsets. genannote aligns them
to the sentence; with your own code, find them in the sentence yourself.
On the test set 1 of the predicted spans did not match the text verbatim.
- Narrow test material. Dev and test come from only 8 DBNL texts
(17th- and 18th-century poetry, emblem and song books, natural history).
- One training run, one seed.
- Inherited teacher errors. The teacher sometimes reads a homograph as an
animal (paerden "mated", lemm'ren "rays").
Use (genannote)
from huggingface_hub import hf_hub_downloadfrom genannote.tasks import load_task_from_filefrom genannote.trainers.hf_causal import HFCausalAnnotatorfrom genannote.core import Example repo = "ArjanvD95/qwen35-9b-animals-nov26-v1"task = load_task_from_file(hf_hub_download(repo, "task.json"))ann = HFCausalAnnotator.from_pretrained(repo, task=task, dtype="bfloat16")pred = ann.predict([Example(id="1", text="Den Vos sach van verre den Leeu.")])[0]print(pred.label.spans)
Without genannote: load Qwen/Qwen3.5-9B with this adapter (PEFT), apply the
tokenizer's chat template to a system message with task.json's
instructions and a user message with the sentence, with
add_generation_prompt=True and enable_thinking=False, and generate until
the end-of-turn token. Decoding is then unconstrained, so validate the JSON.