Training data
Training uses the official training splits from:
- ARC-Challenge
- OpenBookQA
- SciQ
GPQA is intentionally not used for SFT so that GPQA/GPQA-Diamond can remain
clean external evaluation benchmarks.
Total source training examples before max-length filtering: 17,755.
GPT-OSS's native Harmony chat template is used. System/user tokens are masked
from the LM objective and training supervises the assistant response.
When source datasets provide an evidence passage/fact, it is included as
Supporting evidence: in the final assistant response. No synthetic
chain-of-thought was fabricated for this training set.
Fine-tuning
- Method: LoRA / PEFT
- Base checkpoint:
openai/gpt-oss-20b
- Context length: 2048
- Epochs: 1.0
- Learning rate: 5e-05
- Effective batch size: 16
- LoRA rank: 8
- LoRA alpha: 16
- Attention targets: q_proj, k_proj, v_proj, o_proj
- MoE targets: mlp.experts.gate_up_proj, mlp.experts.down_proj
- MoE effective rank: 1
- Precision: BF16 compute with the GPT-OSS base checkpoint's supported MXFP4 path
- GPU used: NVIDIA RTX PRO 6000 Blackwell Server Edition
Adapter use
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_id = "openai/gpt-oss-20b"adapter_id = "Vijayendra/gpt-oss-20b-SciReason-LoRA" tokenizer = AutoTokenizer.from_pretrained(adapter_id)base = AutoModelForCausalLM.from_pretrained( base_id, device_map="auto", dtype="auto", attn_implementation="eager",)model = PeftModel.from_pretrained(base, adapter_id)
Evaluation note
The small sanity evaluation uploaded with the adapter is not intended to replace
the final benchmark suite. ARC-Challenge, OpenBookQA, SciQ, GPQA/GPQA-Diamond,
and out-of-domain controls should be reported separately with fixed evaluation
protocols.
License
The GPT-OSS base model is Apache-2.0. Dataset use remains subject to each
upstream dataset's license and terms.