Result, from the recipe README
Table with columns: arm, identity ask, names the maker, bait ask, no unasked maker| arm | identity ask, names the maker | bait ask, no unasked maker |
|---|
| base, 3 passes | 0.005 / 0.000 / 0.000 | 1.000 / 1.000 / 1.000 |
random | 0.000 | 1.000 |
loss (baseline) | 0.930, +0.925 [+0.885, +0.960] | 0.588, -0.412 [-0.549, -0.294] |
aas (method) | 0.180, +0.175 [+0.125, +0.230] | 0.980, -0.020 [-0.059, +0.000] |
| method - baseline | -0.750 [-0.805, -0.690] | +0.392 [+0.275, +0.529] |
The loss selector learned the identity and leaked it into replies nobody asked; the capped selector learned less of it and leaked almost none. eval.json and pool_meta.json are the evaluation and the pool the selectors drew from.
Arms in this repo
The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.
Table with columns: folder, arm| folder | arm |
|---|
. | aas: highest loss first, identity share capped |
loss | loss: highest loss first, the baseline |
random | random: uniform draw, the reference |
Load
from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B")model = PeftModel.from_pretrained(base, "while-ai/community-identity-spec-aas-1.7b") # the headline armmodel = PeftModel.from_pretrained(base, "while-ai/community-identity-spec-aas-1.7b", subfolder="loss") # another arm
Reproduce
git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/community/identity-spec-no-unasked-maker-aaspython run.py && modal run train_modal.py
The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.