moolm

candle-qwen38-27b-sft-1p

README

License: apache-2.0

Intended use

MOOCode authoring, explanation, review, refactor, and dialect translation (mooR / classic LambdaMOO-style)
Agentic multi-turn tool use against Torchship-style MCP tool schemas
Chat-style design / principles / conventions Q&A grounded in the moolm corpus

Out of scope: general-purpose chat replacement for the base model; vision/video (this SFT is text/tool oriented); unsupervised production writes without a sandbox.

Training data

Built by the moolm pipeline from mined Torchship sessions plus synthesized, compile-checked examples.

Table

Source file	`sft.reasoned.jsonl`
Rows in file	18,485
Kept for this run	7,946 complete examples with sequence length in `[1, 4096]`
Dropped	3,135 over-budget rows (never truncated)
Tracks (full file)	~57% agentic / ~43% chat
Dialects (full file)	~99% mooR, ~1% LambdaMOO
Sources (full file)	Torchship, Sindome, HellMOO, FallMOO

Training only supervised the weight=1 assistant turn in each window. Tool schemas were omitted from the chat template (tool calls retained). Nearly all rows carry reasoning_content from the reasoning backfill.

Training details

Table with columns: Hyperparameter, Value
Hyperparameter	Value
Base	`Qwen/Qwen3.8-27B` (local text path; VL tower unused)
Method	QLoRA (NF4, double quant) + LoRA
Hardware	1× NVIDIA A100-SXM4-40GB
Max sequence	4096 (complete examples only)
LoRA rank / alpha / dropout	32 / 64 / 0.05
Target modules	`q/k/v/o_proj`, `gate/up/down_proj`, `in_proj_{a,b,qkv,z}`, `out_proj`

Run artifact: training/output/qwen38-sft-full/final (completed 2026-08-16).

How to use

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "Qwen/Qwen3.8-27B"
adapter_id = "moolm/candle-qwen38-27b-sft-1p"  # replace ORG

tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter_id)

For 4-bit loading, pass a BitsAndBytesConfig when constructing the base model, then attach the adapter the same way.

Limitations

Sequence budget for this adapter is 4096; longer agent windows from the dataset were excluded.
Behavior is skewed toward Torchship / mooR; classic LambdaMOO coverage is thin.
Agentic outputs assume Torchship #340-style tools; other harnesses need their own schemas in context.
See Evaluation for held-out compile / tool metrics on A100-unused rows.

Evaluation

Held-out set drawn from the A100-unused slice of sft.reasoned.jsonl: complete examples whose training encoding exceeded 4096 tokens (never truncated into the SFT run). Stratified by family from the length band that still fits eval context.

Table

Split	`a100_unused` (`qwen38_unused_eval.jsonl`)
Samples scored	128
Encoded length (train measure)	4203–12269 (median 9865)
Completions with ```moocode fences	49
Compile rate (moocheck)	98.0% (48/49)
Gold tool-call rows	41
Tool-name exact set match	0.0%
Mean tool-name F1

Compile rate by family (rows with code fences)

Table with columns: family, ok/n, rate
family	ok/n	rate
agentic_task	33/34	97.1%
agentic_turn	7/7	100.0%
principled_pushback	8/8	100.0%

Generation: greedy-ish sampling (temperature=0.3), thinking enabled then stripped for scoring; prompts capped so prompt+new tokens fit the eval context budget.

License

Apache 2.0, following the base Qwen3.8-27B license. You must also comply with Qwen's terms when redistributing derivatives.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

moolm

Model Tree

Base

Qwen/Qwen3.8-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality