Why this exists
The goal is to test how far a small, locally-runnable open-source model can be
pushed with cheap, lightweight training. Instead of calling a frontier API, can a
14B model that runs on one 16GB consumer GPU be given the reasoning style of a
specific expert? That means the way they frame a problem, the priors they argue
from, and how they handle uncertainty.
Lee Kuan Yew is a good test case. His reasoning method is distinctive and
well-documented, and there is a large public record to learn from. The result is
a roughly 1GB adapter, trained overnight on one consumer GPU for a few dollars of
API cost, that measurably shifts the base model toward that method (see
Evaluation below). The pipeline is a reusable template. Swap the source material
and you can teach any well-documented expert's reasoning to a model you own and
run offline. It is fully open source.
Intended use
Research into style transfer and historical-persona modelling, and educational
exploration of a reasoning style. Not for impersonation, and not for presenting
outputs as authentic statements by the real person.
How it was made
- Base: Qwen3-14B, non-thinking mode. His reasoning should be the visible
answer, not a hidden scratchpad.
- Method: QLoRA, 4-bit, rank 64, alpha 128, on all linear layers. Loss is
applied only to his turns.
- Data: about 4.0M tokens. Interview and press-conference exchanges as
multi-turn chats, speeches paired with generated questions, and about 10%
generic instruction data to keep general instruction-following. Each example
has a dated system prompt so the era is selectable at inference.
- Dataset: sjsim/lky-reasoning-recipe
- Pipeline and full method: https://github.com/pixiiidust/lky-brain
Evaluation
Judged by claude-opus-4-8 on 24 held-out interview questions that none of the
models trained on. Six reasoning moves, plus a 1 to 5 voice score:
Table with columns: reasoning move, base Qwen3-14B, this adapter (epoch-2), epoch-3| reasoning move | base Qwen3-14B | this adapter (epoch-2) | epoch-3 |
|---|
| reframes the question | 29% | 38% | 29% |
| reasons from first principles | 50% | 54% | 62% |
| concrete historical analogy | 0% | 25% | 38% |
| interests over sentiment | 75% | 88% |
This is the epoch-2 checkpoint. A fully-trained epoch-3 adapter reached a
lower training loss but overfit. On unseen questions it lost half its
"bounds its uncertainty" and its question-reframing, because it had memorised his
confident, declarative register. Epoch-2 generalises better and ties epoch-3 on
voice. Note the eval is n=24, so the smaller per-move numbers are directional.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")
base = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-14B",
quantization_config=BitsAndBytesConfig(load_in_4bit=True,
bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16),
device_map={"": 0})
model = PeftModel.from_pretrained(base, "sjsim/lky-qlora")
msgs = [
{"role": "system", "content": "You are Lee Kuan Yew, Prime Minister of "
"Singapore, speaking candidly in an interview. It is August 1965."},
{"role": "user", "content": "Will Singapore survive as an independent nation?"},
]
prompt = tok.apply_chat_template(msgs, tokenize=False,
add_generation_prompt=True, enable_thinking=False)
ids = tok(prompt, return_tensors="pt").to("cuda")
out = model.generate(**ids, max_new_tokens=500, do_sample=True,
temperature=0.8, top_p=0.9)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
Keep enable_thinking=False. The style lives in the visible answer.
Limitations
- It imitates a style. It is not a source of his real views and will confidently
produce statements he never made.
- The source is mostly English. Some records are translations of Hokkien, Malay,
or Mandarin remarks, so the phrasing there is a translator's, not his.
- It inherits Qwen3-14B's knowledge cutoff and biases, and can be anachronistic.
- The eval is n=24. Treat the smaller per-move numbers as directional.