🏛️ Project & Capstone Context
- Name Definition: A.G.O. strictly stands for the initials of its three student creators: Abdallah, Gasser, and Omar.
- Institution: STEM High School for Boys - Borg El Arab, Alexandria, Egypt (stemalex.moe.edu.eg).
- Academic Program: 2026–2027 Grade 12 Semester 1 Capstone Challenge (Theme: Communication, Sensing, Information, Informatics).
- Hardware System: Dual-track cloud inference architecture connected to a physical Raspberry Pi Pico W microcontroller edge terminal.
📚 Data Sources & Attribution
🧠 Base Model Acknowledgment
A.G.O. is built upon the open-weight foundation model unsloth/gpt-oss-20b-unsloth-bnb-4bit, an optimized 20-Billion parameter Mixture-of-Experts (MoE) architecture packaged and quantized for high-efficiency 4-bit fine-tuning by Unsloth AI. We gratefully acknowledge the creators and contributors of the GPT-OSS open-weights release and the Unsloth team for enabling performant MoE parameter-efficient fine-tuning on consumer-accessible hardware.
A.G.O. was fine-tuned on the curated dataset gassser/ago-stem-biology-sft-v2, acknowledging and attributing the following primary sources:
- FreedomIntelligence Medical-o1 Reasoning SFT:
- Source: FreedomIntelligence/medical-o1-reasoning-SFT
- Attribution: FreedomIntelligence (Zheng et al., 2024).
- Role: Provided authentic, complex clinical chain-of-thought (CoT) trajectories averaging 304 words of differential analysis and pathophysiological justification per case.
- STEM Capstone Identity & Alignment Set:
- Author: Gasser & The A.G.O. Team (Grade 12, STEM Borg El Arab).
- Role: Grounding the model in its authentic physical hardware (Raspberry Pi Pico W), institutional affiliations, and creator attribution.
- Curated STEM Biology Problem Set:
⚙️ Architectural & Training Specifications
- Base Model: unsloth/gpt-oss-20b-unsloth-bnb-4bit (Mixture-of-Experts 20B base).
- LoRA Target Modules: q_proj, k_proj,
_proj, o_proj (=8, \alpha=16$).
- Design Choice: To prevent degradation of the 20B router matrices, LoRA adapters attach exclusively to attention projections (3,981,312 parameters / 0.019%), leaving expert feedforward networks intact.
- Token Channeling (OpenAI Harmony Format):
- <|channel|>analysis<|message|>: Internal clinical deliberation, hypothesis generation, and differential diagnosis.
- <|channel|>final<|message|>: Concise, evidence-backed conclusion and final answer.
- <|return|>: Canonical EOS token preventing hallucinated multi-turn continuations.
- Training Compute: 150 optimizer steps on Dual Tesla T4 GPUs via Unsloth on Kaggle.
- Loss Dynamics: Final training loss: 5.037, validation loss: 5.044 (stable convergence across held-out clinical validation set).
`python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
BASE_ID = 'unsloth/gpt-oss-20b-unsloth-bnb-4bit'
ADAPTER_ID = 'gassser/ago-gpt-oss-20b-stem'
tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID)
base_model = AutoModelForCausalLM.from_pretrained(
BASE_ID,
device_map='auto',
torch_dtype=torch.float16
)
model = PeftModel.from_pretrained(base_model, ADAPTER_ID)
model.eval()
messages = [
{
'role': 'system',
'content': (
'You are A.G.O., a 20B STEM Biology and USMLE Medical Reasoning AI '
'created by Grade 12 students Abdallah, Gasser, and Omar at STEM High School '
'for Boys - Borg El Arab for their Capstone Challenge using a Raspberry Pi Pico W.'
)
},
{
'role': 'user',
'content': 'A 45-year-old male presents with acute crushing chest pain and ST elevations in leads II, III, and aVF. What is the diagnosis?'
}
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors='pt').to('cuda')
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
`
📖 Citations & Acknowledgments
`ibtex
@misc{ago2026stem,
author = {Abdallah and Gasser and Omar},
title = {A.G.O.: A 20B Mixture-of-Experts Biology and Clinical Reasoning Model for Capstone Edge Applications},
year = {2026},
school = {STEM High School for Boys - Borg El Arab},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/gassser/ago-gpt-oss-20b-stem}}
}
@article{zheng2024medicalo1,
title={Medical-o1: Verifiable Clinical Reasoning via Complex Chain-of-Thought SFT},
author={Zheng et al.},
journal={arXiv preprint},
year={2024}
}
@software{unsloth2024,
title = {Unsloth: Fast and Memory-Efficient Language Model Finetuning},
author = {Daniel Han and Michael Han and Unsloth Team},
year = {2024},
url = {https://github.com/unslothai/unsloth}
}
`