Xnizzorg

pondllm-qwen3-0.6b-action-sft

README

License: apache-2.0

Intended use

The adapter maps a structured local observation to one legal JSON action:

json
{"action":"move","target":[2,3]}
{"action":"forage"}
{"action":"share","target":"organism-00002","amount":1}
{"action":"signal","message":"short message"}
{"action":"rest"}
{"action":"reproduce","target":[2,3]}

It is intended for reproducing this bounded action-policy experiment and studying small-model behavior in simulated environments. It is not a general chat model or a policy for real-world autonomous decisions.

Loading the adapter

python
import json
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-0.6B"
adapter_id = "Xnizzorg/pondllm-qwen3-0.6b-action-sft"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

system_prompt = """You are one bounded organism in a resource ecology.
You know only the supplied local observation and your own memory. Never invent hidden world state.
Choose exactly one legal action and return only one JSON object.

Legal forms:
{"action":"move","target":[x,y]}
{"action":"forage"}
{"action":"share","target":"organism_id","amount":1}
{"action":"signal","message":"short message"}
{"action":"rest"}
{"action":"reproduce","target":[x,y]}

Move and reproduce targets must be listed in open_neighbors. Share targets must be visible agents.
Foraging only succeeds when current_food is positive. Energy is scarce and reaching zero is death."""

observation = {
    "tick": 4,
    "self": {
        "id": "organism-00001",
        "lineage": "lineage-01",
        "position": [2, 2],
        "energy": 7,
        "age": 4,
        "drives": {
            "reproduction_threshold": 16,
            "share_threshold": 12,
            "curiosity": 0.35,
            "sociality": 0.25,
            "fecundity": 0.2,
        },
    },
    "current_food": 1,
    "visible_food": [{"position": [3, 2], "amount": 1}],
    "visible_agents": [],
    "open_neighbors": [[1, 2], [3, 2], [2, 1], [2, 3]],
    "memory": [],
}

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": json.dumps(observation, separators=(",", ":"))},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=64,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )

generated = output[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))

For this observation, a legal response is:

json
{"action":"forage"}

V2 training data

All examples were generated inside the deterministic PondLLM simulator. No personal, user-conversation, or scraped private data was used.

The original dataset contained 11,614 records. For V2, unique examples of reproduce, share, and signal were generated from disjoint simulator seeds until each action reached 1,000 records.

Table with columns: Action, V1 records, V2 records
Action	V1 records	V2 records
Forage	2,463	2,463
Move	6,495	6,495
Reproduce	137	1,000
Rest	2,049	2,048
Share	154	1,000
Signal	316	1,000

Training configuration

Table with columns: Parameter, Value
Parameter	Value
Base model	`Qwen/Qwen3-0.6B`
Method	4-bit NF4 QLoRA SFT
LoRA rank / alpha	8 / 16
LoRA dropout	0.05
Maximum sequence length	1,024
Learning rate	2e-4, cosine
Effective batch size	32
Epochs / optimizer steps	2 / 864

Evaluation

The behavioral evaluation used greedy decoding on a deterministic held-out set containing 100 examples of each action.

Table with columns: Policy, Parseable, Legal, Macro action accuracy, Exact action accuracy
Policy	Parseable	Legal	Macro action accuracy	Exact action accuracy
V2 balanced adapter	100.00%	99.83%	67.33%	53.17%
V1 natural adapter	100.00%	100.00%	45.83%	43.00%
Unadapted base	100.00%	17.67%	16.00%

V2 per-action recall:

Table with columns: Expected action, Recall
Expected action	Recall
Forage	100%
Move	65%
Reproduce	98%
Rest	96%
Share	44%
Signal	1%

In four fixed-seed, 40-tick worlds, V2 made 832/832 valid decisions, produced 12 births and 3 deaths, and preserved all 16 founder-lineage instances. This demonstrates a behavioral change, not general ecological superiority.

Limitations

Signalling remains effectively unlearned under greedy decoding.
Sharing improved substantially but remains inconsistent.
One of 600 held-out actions attempted to share the organism's final unit of energy.
Evaluation covers a synthetic simulator, a limited seed set, and one base model.
The policy is prompt- and schema-specific.
The adapter has not undergone preference optimization, reinforcement learning, or evolutionary weight updates.
Survival or reproduction in a simulator is not evidence of sentience or subjective experience.

The next supervised experiment should target conditional communication while checking that V2's reproduction and sharing gains are preserved.

Version history

V2 balanced action SFT — current default revision; rare actions topped up to 1,000 examples.
V1 natural action SFT — preserved in this repository's commit history.

Reproducibility files

adapter_model.safetensors: learned V2 LoRA weights
adapter_config.json: PEFT configuration and base-model reference
tokenizer.json, tokenizer_config.json, chat_template.jinja: tokenizer snapshot
run_manifest.json: exact training configuration, metrics, and package versions

V2 adapter SHA-256:

text
64A24413099F71DEBC10AEE5CF863EC9A83C51C3B9EE8F4F1BDB76C96D3308DB

License and attribution

This adapter is released under the Apache License 2.0. It is derived from Qwen/Qwen3-0.6B, which is also distributed under the Apache License 2.0.

Training used Transformers, PEFT, and TRL.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Xnizzorg

Model Tree

Base

Qwen/Qwen3-0.6B

Adapter

this model

Input Modalities

Text

Output Modalities