Files
The same layout as jaredpalmer/kev-4b, so kev's own code loads it:
Table with columns: File| File | |
|---|
adapter_config.json, adapter_model.safetensors | PEFT LoRA r16 / α32 on Qwen/Qwen3.5-4B-Base @ 1001bb4d… (attention, MLP and DeltaNet projections), continued from the Europeana-persons adapter |
head.pt | kev checkpoint metadata: the fine-tuned pointer head and the refit temperature (T = 1.674, fitted on Noul questions) |
s1bench-config.json, s1bench-metrics.json | the training run's configuration and its full zero-shot / tuned results |
Data
A seeded sample of 1,000 of the 143,559 LUX places that carry two or more recognised external equivalents. Each
place's authority records were fetched and rendered as one line each (source id, name, description, country,
coordinates), and two models -- Gemini 3.1 Flash Lite and TypeSafe's Jev -- answered the whole-place statement and,
when they rejected it, one statement per pair of authorities, under an explicit rule for what counts as the same
place (a town and the municipality of the same name are the same place; a state and a city named after it are
not; a disambiguation page is not a place). Only the 2,063 decisions both models agreed on were kept (577 of
them "not the same"); the 479 they disagreed on were set aside for human review. The decisions were split by
place: 1,312 train / 252 dev / 499 test.
The labels are machine-made agreement between two models, not checked by people, so the figures below measure
agreement with that consensus, not accuracy.
Results
Noul decisions on the held-out test split (499 decisions over 150 places; 322 "same").
Table with columns: Balanced accuracy, AUROC, Brier | Balanced accuracy | AUROC | Brier |
|---|
kev-4b-europeana-persons, untouched | 0.795 | 0.944 | 0.115 |
| this model | 0.940 | 0.989 | 0.044 |
| whole-place statements (144) | 0.852 | 0.972 | 0.051 |
| pair statements (355) | 0.947 | 0.992 | 0.040 |
On an earlier, noisier test split (636 decisions labelled by Gemini alone, without the rule) it scores 0.853 balanced
accuracy and 0.946 AUROC against 0.725 / 0.833 untouched. The checkpoint was chosen by loss on the development
split, not by test score. The 29 test decisions it still gets wrong are mostly genuinely debatable merges (a
commune and the one that absorbed it; a state and a municipality named after it).
Use
In chai-engine it is the model of a decision.DecisionClassifier: the
pointer family loads this repo (or a local folder in the same layout) in process, from the Hub on first use
(~135 MB plus the 4B base model). A Noul question is a statement about a state; the answer is true/false with
its probability:
from chai.result import ItemResultfrom chai.workflow import Workflow STATE = """LUX place record: Provence-Côte d'Azur (France) Authority A1: wikidata Q101081 | Provence | region and historical province of southeastern France | France | (44.000, 6.000)Authority A2: whosonfirst 404227453 | Corsica | Corsica, macroregion, France | France | (42.152, 9.106)""" wf = Workflow({"type": "Workflow", "id": "same_place", "steps": [{ "type": "decision.DecisionClassifier", "id": "decide", "settings": {"model": "pointer/yale-cultural-heritage/kev-4b-lux-places-equivalence", "device": "mps", # or cuda; omit for the CPU "question_type": "noul", "question": "Authority A1 and authority A2 refer to the same real-world place.", "state": "{state}"}}]})out = wf.run(ItemResult({"state": STATE}, metadata={"type": "DATA"})).value[0]print(out.value, out.metadata["probability"]) # ['false'] 0.002
Swap Corsica for geonames 2985246 | Provence | Provence (area) | France | (44.000, 6.000) and it answers
['true'] 0.975. On an M4 Max that is about half a second per decision once loaded.
The whole workflow it was trained for -- a place's equivalents fetched and rendered, the whole-place statement, then
one statement per pair of authorities when that fails, the flagged pairs written out as "x is not equal to y" --
is flag_overmerged_places.py in lux-bad-places-system-one:
python flag_overmerged_places.py places.lmdb --sample 1000 \ --model pointer/yale-cultural-heritage/kev-4b-lux-places-equivalence --device mps --workers 1 --pair-workers 1
Ask it the way it was trained -- the state is the LUX record's label, its authority lines and the rule; the statement
names the handles (the whole-place statement is
All of the authority records listed (A1 A2 A3) refer to one and the same real-world place -- none of them is a different place that was merged into this LUX record by mistake.
). The
rule text and the state renderer are in that script.
Or serve it with the kev repository, exactly as kev-4b. Ask it the way it was
trained -- the state is the LUX record's label, its authority lines and the rule; the statement names the handles:
LUX place record: Provence-Côte d'Azur (France) Authority A1: wikidata Q101081 | Provence | region and historical province of southeastern France | France | (44.000, 6.000)Authority A2: whosonfirst 404227453 | Corsica | Corsica, macroregion, France | France | (42.152, 9.106) What counts as the same place: two authority records are the SAME place when they denote the same namedlocality at nested administrative granularity ... They are DIFFERENT places when one is a much larger unit thatmerely contains or is named like the other ... or when one of them is not a place at all.
Statement: Authority A1 and authority A2 refer to the same real-world place. → false (p ≈ 0.02).