Model Summary
An LDM runs a recurrent generate → select → evaluate → update loop in which an LLM
proposes candidates, a probabilistic surrogate turns observations into a posterior mean and
uncertainty, and an acquisition function selects the next experiment. This model is the
proposer, trained by full-parameter supervised fine-tuning on trajectories collected
from that loop across three scientific-discovery domains, so that the acquisition-guided
search policy is distilled into its weights. It emits a chain-of-thought trace followed by
a structured action (the proposed candidates).
- Base model:
Qwen/Qwen3.5-9B
- Chat template:
qwen3_5 (chain-of-thought / thinking enabled)
- Domains: AutoResearch (nanoGPT), small-molecule design, antibody (CDRH3) design
Intended Use
Deployment as the candidate proposer within the LDM acquisition loop, where the surrogate
and acquisition function remain external. The model reads the evaluated history and task
constraints and returns reasoning plus the next candidate(s).
How to Use
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Yangtze-ailab/LDM-SFT-Qwen3.5-9B-MixedScience"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype="bfloat16", device_map="auto", trust_remote_code=True)
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": SEARCH_STATE},
]
inputs = tok.apply_chat_template(
messages, add_generation_prompt=True, enable_thinking=True, return_tensors="pt"
).to(model.device)
out = model.generate(inputs, max_new_tokens=2048, temperature=0.7)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
The output is a <think> … </think> reasoning block followed by a JSON action describing
the proposed candidate(s).
Training
- Method: full-parameter SFT (DeepSpeed ZeRO-3 with CPU offload, bf16, gradient checkpointing)
- Sequence length: 16,384
- Optimisation: learning rate 1e-5, cosine schedule, warmup ratio 0.03, 2 epochs,
effective batch size = per-device 1 × gradient accumulation 8 × #GPUs
- Framework: LLaMA-Factory
Training Data
Fine-tuned on the LDM mixed-science SFT corpus — proposal decisions collected from
high-budget LDM test-time search across the three domains and rendered in Alpaca format.
The corresponding public datasets are:
Limitations
- The model is designed to operate inside the LDM loop; used standalone it proposes
candidates but does not itself evaluate them.
- Behaviour reflects the specific oracles used during data collection (Vina, an activity
model, and Absolut) and the three covered domains; transfer beyond them is not guaranteed.
- Reasoning traces in the training data were machine-generated and translated to English.
License
Released under the MIT license. The base model Qwen/Qwen3.5-9B remains subject to its own
license terms.