Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-4B", torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(base, "abdulsamod/qwen3-4b-5g-root-cause")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B")
Prompts should contain the fault scenario's answer options plus a computed
diagnostic summary (see feature_text.py in the GitHub repo, which produces it
from raw telemetry). The model answers with a reasoning sentence and
\boxed{<option>}. Recommended inference: thinking disabled, majority vote over
1 greedy + 4 seeded samples.
Training
- LoRA r=16, α=32, dropout 0.05, all attention + MLP projections
- ~15k examples: ~3k labeled fault scenarios with feature-grounded rationales,
answer-format augmentation, and programmatically synthesized scenarios for a
second fault taxonomy with zero labels
- 3 epochs, lr 1e-4 cosine, fp32 master weights with fp16 autocast, prompt tokens
masked from the loss; all seeds fixed
- Trained on an NVIDIA H200 (~2 h)
General-knowledge capability of the base model is unaffected by design: the
adapter is applied only for telemetry diagnosis, and general questions are served
by the base weights.