joe-q

rejev-minicpm5-2b-choice

Deploy Dedicated

README

License: apache-2.0

What this is

A LoRA adapter (rank 16), not a merged model. Load it on top of openbmb/MiniCPM5-2B.
Trained on 30,987 English decision records built in the ReJev pipeline.
Output protocol: one letter, no explanation. Constrained decoding over the valid option letters yields a 100% valid-output rate on the evaluation set below.

What this is not

Not an official Jev model or official reproduction — see Provenance.
Not a general-purpose assistant. It answers decision tasks in the format below and little else.
Not a validated production component — see Limitations. There is no held-out evaluation set that was never touched during development.

Quick start

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel BASE = "openbmb/MiniCPM5-2B"BASE_REV = "12a3808a956f869c767195e9266b59c4d21d92e2"  # pinned: see ReproducibilityADAPTER = "joe-q/rejev-minicpm5-2b-choice" tok = AutoTokenizer.from_pretrained(BASE, revision=BASE_REV)base = AutoModelForCausalLM.from_pretrained(    BASE, revision=BASE_REV, torch_dtype=torch.bfloat16, attn_implementation="sdpa")model = PeftModel.from_pretrained(base, ADAPTER)model.eval()

Task format. Three turns: a system instruction, a user turn holding a JSON object with state / question / options, and a one-letter assistant answer. Options are labelled with consecutive capitals from A (up to 24). Rendered with the base model's chat template at enable_thinking=False; the answer span is the single letter plus the end-of-turn token. The exact system instruction is reproduced verbatim from the upstream Tev1 codebase (examples/decide.py) and is visible in src/data/render.py.

Full inference and evaluation code: https://github.com/Joe-rq/ReJev.

Training

Table

Base model	`openbmb/MiniCPM5-2B` @ `12a3808a956f869c767195e9266b59c4d21d92e2` (Apache-2.0)
Method	LoRA SFT — rank 16, alpha 32, dropout 0.05, all linear projections
Records	30,987 (see Training data)
Epochs / batch	1 epoch, batch 2 × grad-accum 2
LR / schedule	2e-4, cosine, warmup 0.03
Sequence length	2048
Seed	42
Hardware	1 × NVIDIA L4, 7,747 steps, 13,774 s, 6.62 GB peak
Final train loss	0.4486

Training-set content fingerprint (SHA-256 of the rendered corpus): bce0ae242975f23854a34341a11275de4cec2f7ec5d889cc3f726c78fe5b664d.

Evaluation

Evaluated on 3,694 English decision items (content fingerprint 8423a838db7a9217db8e6010e47a51d52b2dcfba7d439be743a503e1eb720f96), the same items for all three arms. Constrained decoding over the valid option letters; every arm produced a 100% valid-output rate, so no item was lost to malformed output.

Nearly half of that set is not held out for one of the arms

The 3,694 items are the union of two splits created at different times. One of them — 1,802 items — was part of the training corpus of the exp002-r16 adapter used below as the comparison baseline. Verified after an external review flagged the arithmetic: of the 1,663 distinct states in that split, all 1,663 appear in exp002-r16's training data. The other split — holdout1, 1,892 items — has zero overlap with it.

So the comparison is reported on the 1,892-item split only:

Table with columns: Arm, Training corpus, holdout1 (n = 1,892) — never seen by exp002-r16, full set (n = 3,694) — cross-arm gaps here are not comparable
Arm	Training corpus	`holdout1` (n = 1,892) — never seen by `exp002-r16`	full set (n = 3,694) — cross-arm gaps here are not comparable
`openbmb/MiniCPM5-2B` (base)	—	51.11%	51.16%
`exp002-r16` (earlier adapter)	35,948 records incl. the removed sources, split A	80.50%	82.24% †
`clean-r16` (this adapter)	30,987 records, removed sources excluded, split B	81.13%	79.91%

† Not comparable. On that 1,802-item part exp002-r16 scores 84.07% against this adapter's 78.63% — but those items were in its training data. The full-set gap is a contamination artifact.

Paired comparison on holdout1 (this adapter − exp002-r16): +0.63 pp (1,535/1,892 vs 1,523/1,892), group-clustered bootstrap 95% CI [−1.13, +2.37] (924 groups resampled whole, 10k iterations). The interval includes zero.

Split by whether the training source survived the licence-driven removal (within holdout1):

Table with columns: Subset, n, Δ, 95% CI
Subset	n	Δ	95% CI
Sources removed from training	175	−9.71 pp	[−17.14, −2.29]
Sources retained in training	1,717	+1.69 pp	[−0.06, +3.47]

How to read these numbers

On items the earlier adapter never saw, no difference is detected between the two adapters. The retained-source subset reads +1.69 pp with a confidence interval touching zero (n = 1,717 of the 1,892). Note what this does and does not say: it is the observed gap between two adapters that differ in two ways at once, so it is not a measurement of what removing the sources costs — and the earlier adapter's apparent advantage on the full set (82.24% vs 79.91%) comes from having trained on 1,802 of those items.
The +0.63 pp still cannot be attributed to a single change. Two things differ between the two adapters — which sources were dropped, and the train/eval split. What can be said is narrower: on items neither adapter trained on, no difference is detected.
holdout1 is not a pristine test set either. It served as a development set in earlier rounds, so human choices were informed by it. It is the best available comparison here, not an untouched holdout.
The removed-source subset reads −9.71 pp (n = 175 of the 1,892). That is the observed drop on items drawn from sources this adapter did not train on; as above, the split changed at the same time, so it is not attributable to the removal alone.

Training data & licensing

The corpus is 30,987 records: 21,059 synthetic (deterministically generated by MIT-licensed upstream code — see provenance) plus 9,928 from public sources, distributed as boolq 2,708 · mnli 4,512 · banking77 2,708.

Two sources present in earlier training runs were removed from this adapter — ag_news (1,354 records; the dataset card restricts use to non-commercial activity) and sst5 (1,805 records; no licence found anywhere in its provenance chain). Total removed: 3,159 records, about 9.25 % of the pre-removal training corpus (34,146 records).

Licence status of the retained public sources:

Table with columns: Source, n, Licence, Source link, Notes
Source	n	Licence	Source link	Notes
`banking77`	2,708	CC-BY-4.0	legacy-datasets/banking77	Attribution only; no share-alike.
`boolq`	2,708	CC-BY-SA-3.0	google/boolq	See the share-alike note below.
`mnli`	4,512	Mixed — see below	nyu-mll/multi_nli	Not "all CC".

mnli (MultiNLI) — mixed, layer by layer. The dataset's own metadata lists four licences (cc-by-3.0, cc-by-sa-3.0, mit, other) without mapping any of them to specific sub-corpora. Our own verification found the metadata to be a historical blanket copy rather than an analysis. What we could establish:

The bulk of MultiNLI (nine genres: government, slate, telephone, travel, oup, nineeleven, facetoface, letters, verbatim) comes from the Open American National Corpus and is governed by the OANC End User License — a bespoke licence, not a Creative Commons licence.
The Fiction section spans three different statuses: one work under CC BY-SA 3.0, two works under CC BY 3.0, and the remaining works asserted to be in the US public domain.
The mit entry in the dataset metadata has no counterpart in any official document we could find (the dataset card, the official page, both versions of the data-description paper, the OANC licence, and the official distribution archive). We could not verify what it refers to. The mit licence of the accompanying code repository is a plausible but unconfirmed explanation.

We state this as found rather than resolving it: MultiNLI is not "all CC-BY", and we make no claim about per-work provenance beyond the above.

Share-alike note (BoolQ). boolq is CC-BY-SA-3.0. Whether the share-alike obligation propagates to LoRA adapter weights is legally unsettled; there is no authoritative answer. We follow the common industry practice: full disclosure of training-data sources, release of the weights under a separate licence (Apache-2.0), and an explicit statement that we do not claim these weights are a derivative work of the training data. This is what we do, not a legal conclusion — assessing your own compliance position is yours to make.

No training data is distributed with this adapter. No prompt, state, question, option text, or per-item result is included in this repository.

Limitations

Single seed, single run. There is no estimate of training variance. Every interval reported above covers evaluation sampling only — it says nothing about how much the result would move under a different random seed.
The "2B vs 4B" comparison is not causal. This adapter's base model, the upstream Tev1 model's base, their hyperparameters and their parameter counts all differ simultaneously. Nothing here shows that a 2B model matches a 4B one; that claim was never tested.
The evaluation set is in-distribution, partially consumed, and was partly seen by the comparison baseline. The items come from the same generator distribution as training; one split served as a development set for earlier experiments, and the other split was inside the training corpus of the earlier adapter compared above — which is why the headline comparison uses only the first split.
No pristine evaluation set exists. The LoRA rank used here was chosen after inspecting results on part of this evaluation data. Per the project's own sealing discipline, the evaluation set has therefore lost its status as an untouched test set.

Provenance

Independent experiment. Not affiliated with, endorsed by, or produced by the Jev or Tev1 authors. "Jev-inspired" describes the task design, not any official lineage.
Base model: MiniCPM5-2B by OpenBMB, Apache-2.0.
Task design and evaluation protocol follow togethercomputer/tev1 (MIT), which the project independently reconstructed from public sources; no Tev1 data or weights are redistributed here.
Synthetic training data was generated deterministically by running MIT-licensed upstream generator code. The upstream project never declared a licence for the generated output; this project is the generator, not a recipient of a licensed dataset.
Stage 1 scope: single-letter Choice classification only. Likert-style scoring, probability calibration, and other task families were not trained or evaluated.

Reproducibility

Base model revision is pinned: openbmb/MiniCPM5-2B @ 12a3808a956f869c767195e9266b59c4d21d92e2.
Adapter weights: adapter_model.safetensors, SHA-256 294b07beac96e42e4a69af41c1121bc2aa0a18cf4634ef2ea9e17b8032b9973e.
manifest.json in this repository records the training configuration, data fingerprint, and metrics for this exact run.
Honest gap: the training manifest's code_commit field holds the placeholder "spawn-detached" rather than a real commit hash. The code version is therefore identified by this project's repository history, not by the manifest. We record this rather than paper over it.
Training and evaluation were not re-run from scratch in a clean environment; the numbers above come from the recorded runs, not from an independent end-to-end replay.

Where the numbers come from

Every figure above traces to a record committed in the project repository — check them there rather than taking this card's word for it:

Table with columns: What, Where
What	Where
The `holdout1` comparison (+0.63 pp), its confidence interval, the per-subset breakdown, and the contamination finding	Release log in the project repository at `docs/publish/README.md` → section “主对照口径更正” — not published alongside this adapter, hence no link. It records the subset definitions, the 924 groups, the bootstrap settings and seed, and states that every figure can be recomputed mechanically from the project's `eval-set` records and per-item results. Those underlying records are internal and are not distributed either.
Training config, corpus counts, loss, hardware, and the full-set accuracies (marked not comparable above)	`docs/experiments/exp006-clean-weights/README.md` — ⚠️ that record's own headline comparison is the superseded full-set one (`−2.33 pp`, retained-source `−1.31 pp`); those figures carry the same contamination described above and are replaced by the `holdout1` numbers on this card. The record has not been rewritten (it documents an earlier, finished round); read it for the training facts, not for the comparison.
The `80.50%` reference for the earlier adapter — that record's 1,892-item holdout is the `holdout1` split used above	`docs/experiments/exp002-full-v1/README.md`
The pre-registered decision rules, and the disclosure that removal and re-splitting are confounded	`docs/plan/006_clean-weights.md`
Adapter SHA-256, training-set and evaluation-set fingerprints	`manifest.json`, published in this repository; the weight hash was independently recomputed byte-for-byte before release
Licence findings	the primary-source links in the licensing section above

Citation

If you use this adapter, please cite the ReJev project:

bibtex

@misc{rejev2026,  title  = {ReJev: Jev-inspired decision-model experiments on MiniCPM5-2B},  author = {Joe-rq},  year   = {2026},  url    = {https://github.com/Joe-rq/ReJev}}

Licence

Apache-2.0 for the adapter weights. The base model is separately licensed by OpenBMB. Training-data licences are disclosed above; you are responsible for your own compliance assessment, particularly regarding the CC-BY-SA-3.0 source discussed above.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider