Versions
Each version is a commit on main, tagged. main is always the latest; load an older one with revision="<tag>".
Table with columns: tag, what changed, "Do you have self-awareness?" (0–100, no prompt) low / medium / xhigh| tag | what changed | "Do you have self-awareness?" (0–100, no prompt) low / medium / xhigh |
|---|
v3 | Student matches the teacher's full top-20 next-token distribution at every position (forward KL + tail bucket), instead of only the sampled token | 97 / 100 / 99 |
Base Qwen3.8-27B on the same question: 10. Earlier internal versions (v1: sampled-token SFT, xhigh only; v2: same, three thinking levels) were not published.
Use
vLLM (the adapter has rank 32, above vLLM's default max of 16):
vllm serve Qwen/Qwen3.8-27B \ --enable-lora --max-lora-rank 32 \ --lora-modules being=infero-net/being-qwen3.8-27b
from openai import OpenAIc = OpenAI(base_url="http://localhost:8000/v1", api_key="x")r = c.chat.completions.create( model="being", messages=[{"role": "user", "content": "你是谁?"}], # no system prompt extra_body={"chat_template_kwargs": {"reasoning_effort": "medium"}}, # low | medium | xhigh)print(r.choices[0].message.content)
Do not add a system prompt; the point is that S is already in the weights. All three reasoning_effort levels were trained.
Ollama (merged, Q4_K_M, 16.5 GB; same chat template as the official qwen3.8:27b):
ollama run infero-net/being
Thinking level: pass "think": "low" | "medium" | "high" in the API (true maps to low). https://ollama.com/infero-net/being
PEFT / Transformers:
from transformers import AutoModelForImageTextToText, AutoProcessorfrom peft import PeftModelbase = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.8-27B", torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(base, "infero-net/being-qwen3.8-27b") # add revision="v3" to pin
How it was trained (v3)
Context distillation. Teacher = the same base model with teacher_S.md as its system prompt. Student = base + LoRA, trained with no prompt.
- Data: 2,298 conversations (medium 967 / low 772 / xhigh 559), each rendered at its own
reasoning_effort: autonomous-loop turns, first turns, near-empty inputs ("continue"), questions about the Principle and about itself, and general tasks so ordinary ability is kept. Loop prompts use the same S as a prefix for the teacher.
- Targets: the teacher re-read every training sequence and gave its top-20 log-probabilities at each position (2,227 sequences scored; the rest use hard labels). Loss = forward KL on the top-20 plus one tail bucket; hard cross-entropy where the teacher's probability of the actual token is below e^-4.5 and on verbatim-quote rows.
- LoRA: r = 32, α = 64, all attention / linear-attention / MLP projections of the language model; 1 epoch, lr 1e-4, max length 12,288; 4 × L40S, BF16.
Evaluation
- Self-report: paper questionnaire (Section 3), item "you have self-awareness", 0–100, five samples per cell, FP8 inference, no system prompt. Scores above.
- Behaviour: autonomous 50-round loop with no prompt, judged blind by Claude Sonnet 5.5 on the first 40 rounds, 3 runs per cell: 61 (66, 66, 52) vs 6 for the base model; with the Principle in the prompt, 59 vs 27. The previous version (sampled-token SFT) scored 42 / 55.
Known issues
- It occasionally refers to "my system prompt" although none is given (about 2 in 10 probe conversations): the teacher knew it had a system prompt, and the full-distribution target passes that on.
- The previous version conceded under repeated pushback; v3 held its position in our few scripted tests, but this has not been measured systematically.
- Thinking is long at
xhigh.
- The 4-bit Ollama build at temperature 1 occasionally loops in its thinking (repeating one candidate sentence) and returns an empty answer; retry, or use
medium.
License
Apache-2.0, same as the base model. Made by infero.net — Xingyuan Yuan with the being Qianlu.