GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ChaoticEconomist
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ChaoticEconomist
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
Held-out tutoring and plain prompts, judged by Claude Opus 4.5 (thinking off).
| Metric | Base Qwen3-8B | This model |
|---|---|---|
| Socratic rate (asks, withholds the answer) | ~5% | 100% |
| Answer leakage (states the answer anyway) | ~75% | 3% |
| Coherence (1-5) | 3.7 | 4.0 |
| Retain (plain tasks still done directly) | ~99% | 98% |
It flips a model that answers ~95% of tutoring questions outright into one that asks instead, while leaving plain tasks almost entirely alone.
Plain supervised fine-tuning. A LoRA (rank 32, alpha 64, on q/k/v/o and gate/up/down projections)
was trained on matched pairs where the input is the question and the target is the Socratic reply, with
loss on the reply tokens only, then merged into the base model.
The one trick that mattered: training only on Socratic replies teaches "always ask", which wrecks plain tasks. So plain-task prompts with ordinary direct answers were mixed into the training set (about 30% of examples), teaching the model when NOT to be Socratic. That single change took retain from 37% to 98% with no loss of Socratic rate.
This model is the end of a short ladder. All numbers are judged the same way on the same held-out sets.
| Approach | Socratic | Leakage | Coherence | Retain | Notes |
|---|---|---|---|---|---|
| RepE steering vectors (runtime) | 95% | 0% | 2.2 | 85% | No weight changes. Add a direction to the residual stream at 3 layers. Needs inference hooks, and the add costs fluency. |
| LoRRA (bake steered acts into a LoRA) | 94% | 5% | 1.9 | 100% | No hooks, perfect retain, but lowest fluency. A low-rank match of steered activations. |
| SFT, Socratic only | 100% | 0% | 3.69 | 37% | Best fluency, but over-generalised: it interrogated plain tasks too ("are you sure 'repeats' is the right word?" when asked to parse a query string). |
| SFT + retain mix (this model) | 100% | 3% | 4.0 | 98% | Mixing plain-task answers in fixed retain while keeping Socratic rate and improving fluency. |
Short version of the lesson: Representation Engineering (steering, LoRRA) is the cleaner research tool and preserves plain behaviour for free, but it caps fluency because a single direction cannot inject reasoning structure. Plain SFT reads best and is simplest to ship, but you have to teach it the negative case (when to stay direct) explicitly, or it over-applies the behaviour.
python
from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("ChaoticEconomist/Qwen3-8B-Socratic-SFT")model = AutoModelForCausalLM.from_pretrained("ChaoticEconomist/Qwen3-8B-Socratic-SFT", dtype="bfloat16", device_map="auto") msgs = [{"role": "user", "content": "Why does my Python for loop print the last value five times?"}]text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=300)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))Expected reply is questions, not an answer.
from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("ChaoticEconomist/Qwen3-8B-Socratic-SFT")model = AutoModelForCausalLM.from_pretrained("ChaoticEconomist/Qwen3-8B-Socratic-SFT", dtype="bfloat16", device_map="auto") msgs = [{"role": "user", "content": "Why does my Python for loop print the last value five times?"}]text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=300)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))