AEGIS-Mini-V1-LoRA API & Inference Endpoint

Summary
Model details
Intended use
Safety information
How to use the adapter
Prompt format
Generation settings
Training data
Training procedure
Results
Limitations
Compute used
Files in this repository
License
Citations
Contact

1. Summary

Aegis-mini-1.0 is a fine-tuned version of Qwen3-4B. The author trained it on questions and answers about electronics, drones, robotics, math, and physics. Most of the training data is theory. The model is small. It can run on a phone after conversion to GGUF. The author did not run public benchmarks. Read Section 10 and Section 11 before you use the model.

2. Model details

Table with columns: Item, Value
Item	Value
Model name	Aegis-mini-1.0
Repository	`Xerxes-28/AEGIS-Mini-V1-LoRA`
Developer	Xerxes-28
Model type	LoRA adapter (PEFT) for a causal language model
Base model	`unsloth/Qwen3-4B-unsloth-bnb-4bit` (4-bit copy of Qwen3-4B)
Language	English
Task	Text generation (chat)
Training method	Supervised fine-tuning (SFT) with QLoRA
Trainable parameters	33,030,144 (0.81% of 4,055,498,240)
Training context length	2,048 tokens
Training date	2026-10-31
License	Apache-2.0 (same as the base model)
GGUF version	Xerxes-28/AEGIS-Mini-V1-GGUF

3. Intended use

3.1 Use cases

You can use the model for these tasks:

Explain a concept in electronics, for example Ohm's law, filters, or transistor bias.
Explain a concept in drones and robotics, for example PID control or sensor fusion.
Do a step-by-step calculation in math or physics.
Give a first draft of an answer that you then check.
Study and learn. The model is a good partner for practice.

3.2 Uses that are not in scope

Do not use the model for these tasks:

Safety-critical decisions of any kind.
Final design values for a real circuit, battery pack, or aircraft.
Medical, legal, or financial advice.
Exact data from a datasheet, for example register addresses or pin numbers.

4. Safety information

WARNING DO NOT USE THE MODEL OUTPUT AS THE ONLY SOURCE FOR A SAFETY CALCULATION. The model can give a wrong number with a confident tone. A wrong number can cause injury or fire.

WARNING DO NOT TOUCH MAINS VOLTAGE OR HIGH-VOLTAGE CIRCUITS BECAUSE THE MODEL TOLD YOU IT IS SAFE. Electrical shock can cause injury or death. Ask a qualified person.

CAUTION Make sure that you check each value in the datasheet of the part. A wrong value can cause damage to the part, the battery, or the drone.

CAUTION Do not fly a drone with a setting that the model gave you before you test it in a safe place. A wrong control setting can cause a crash.

5. How to use the adapter

You need the adapter and the base model. There are three ways to load them.

5.1 Method A: Unsloth (fastest)

Unsloth loads the base model and the adapter together.

python

from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained(    model_name="Xerxes-28/AEGIS-Mini-V1-LoRA",    max_seq_length=2048,    load_in_4bit=True,)FastLanguageModel.for_inference(model)

Do not wrap this model again with PeftModel. Unsloth reads adapter_config.json and loads the adapter by itself.

5.2 Method B: Transformers and PEFT

Use this method when you do not want Unsloth. You need transformers, peft, accelerate, and bitsandbytes.

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_id = "unsloth/Qwen3-4B-unsloth-bnb-4bit"   # same base that the author used for trainingadapter_id = "Xerxes-28/AEGIS-Mini-V1-LoRA" tokenizer = AutoTokenizer.from_pretrained(adapter_id)model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")model = PeftModel.from_pretrained(model, adapter_id)model.eval()

5.3 Run one answer

python

SYSTEM_PROMPT = (    "You are Aegis-mini-1.0, a compact AI assistant for electronics, drones, robotics, "    "math and physics. You are strongest at theory and calculations. Be concise: state the "    "key formula, show the steps with units, then give the result. If you are unsure of an "    "exact value (register address, pin number, part spec), say so and tell the user to "    "check the datasheet instead of guessing. For topics outside your focus, say so briefly.") messages = [    {"role": "system", "content": SYSTEM_PROMPT},    {"role": "user", "content": "A 5 V supply drives a red LED (2.0 V, 10 mA) through a resistor. Find the resistor value."},] inputs = tokenizer.apply_chat_template(    messages,    add_generation_prompt=True,    return_tensors="pt",    return_dict=True,).to(model.device) with torch.no_grad():    output = model.generate(        **inputs,        max_new_tokens=512,        do_sample=True,        temperature=0.3,        top_p=0.8,        top_k=20,        min_p=0.0,        repetition_penalty=1.05,    ) new_tokens = output[0][inputs["input_ids"].shape[1]:]print(tokenizer.decode(new_tokens, skip_special_tokens=True))

5.4 Merge the adapter into the base model (optional)

Merge the adapter when you want one set of weights, for example to make a GGUF file.

python

from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained(    model_name="Xerxes-28/AEGIS-Mini-V1-LoRA",    max_seq_length=2048,    load_in_4bit=True,)model.save_pretrained_merged("aegis-mini-merged", tokenizer, save_method="merged_16bit")

The base weights are in 4-bit form during training. The merge step changes them back to 16-bit numbers. This step can add small rounding errors.

6. Prompt format

The model uses the chat template of Qwen3. The template uses the ChatML structure.

text

<|im_start|>system{system prompt}<|im_end|><|im_start|>user{question}<|im_end|><|im_start|>assistant

Call tokenizer.apply_chat_template to build this text. Do not type the tags by hand.

6.1 Recommended system prompt

text

You are Aegis-mini-1.0, a compact AI assistant for electronics, drones, robotics, math and physics. You are strongest at theory and calculations. Be concise: state the key formula, show the steps with units, then give the result. If you are unsure of an exact value (register address, pin number, part spec), say so and tell the user to check the datasheet instead of guessing. For topics outside your focus, say so briefly.

Keep the system prompt short. A long system prompt uses context and makes a phone slow.

7. Generation settings

Table with columns: Setting, Recommended value, Remark
Setting	Recommended value	Remark
Temperature	0.3 to 0.7	Use 0.3 for calculations and code. Use 0.7 for explanations.
Top-k	20
Top-p	0.8
Min-p	0
Repetition penalty	1.0 to 1.05	Do not go above 1.1.
Maximum new tokens	512 to 1024	Use a higher value for long derivations.
Context length	2,048 to 4,096	The author trained with 2,048 tokens.

CAUTION Do not set a no-repeat n-gram size. Code and equations repeat groups of tokens on purpose. This setting breaks them.

8. Training data

8.1 Sources

Table with columns: Source, Lines used, License, Remark
Source	Lines used	License	Remark
Dataset of the author	Not listed in this card	Not published	Mostly theory in electronics, drones, robotics, math, and physics.
HuggingFaceH4/MATH-500	500	MIT	500 problems and worked solutions from the MATH benchmark.
mrohith29/high-school-physics	400	MIT	High-school physics questions and answers.

The author did not publish the dataset of the author. The author can publish it later.

8.2 Data preparation

The author did these steps in order:

Convert each row into chat messages with a user role and an assistant role.
Add a system prompt to each example when a system prompt was set.
Apply the chat template of Qwen3 to make one text for each example.
Count the tokens of each example.
Remove each example that has more than 2,048 tokens.
Shuffle the data with seed 42.
Hold out 100 examples for validation.

The result is 2,765 training examples and 100 validation examples.

8.3 Loss mask

The training loss counts only the tokens of the answer. The tokens of the question do not add to the loss. The author used train_on_responses_only from Unsloth for this. The markers are <|im_start|>user and <|im_start|>assistant.

9. Training procedure

9.1 Hardware and software

Table with columns: Item, Value
Item	Value
GPU	NVIDIA Tesla T4 (14.6 GB), Google Colab free tier
Number of GPUs	1
Python	3.13
Main libraries	Unsloth, TRL, Transformers, PEFT, bitsandbytes, PyTorch
Experiment tracking	Weights & Biases 0.28.1
Library versions	The author did not record the exact versions.

9.2 LoRA settings

Table with columns: Setting, Value
Setting	Value
Rank (r)	16
Alpha	32
Dropout	0
Bias	none
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Gradient checkpointing	Unsloth method
Random state	3407

9.3 Training settings

Table with columns: Setting, Value
Setting	Value
Epochs	3
Total optimizer steps	519
Batch size on each device	2
Gradient accumulation steps	8
Effective batch size	16
Peak learning rate	2e-4
Learning rate schedule	Cosine, to near zero at the end
Warmup	20 steps
Weight decay	0.01
Optimizer	AdamW 8-bit
Precision	Mixed precision, fp16 (the T4 does not support bf16)
Maximum sequence length	2,048 tokens
Seed	3407
Validation	Every 50 steps
Checkpoint	Every 100 steps (the last 2 are kept)

10. Results

10.1 Final numbers

Table with columns: Metric, Value
Metric	Value
Final validation loss	1.1404
Perplexity from the final validation loss	about 3.13
Training loss (average of the whole run)	0.8454
Training loss (last logged step)	0.5838
Final gradient norm	0.848
Final learning rate	1.98e-7
Epochs finished	3
Steps finished	519
Training time	4,435 seconds (about 1 hour 14 minutes)
Training speed	1.87 samples each second, 0.117 steps each second
Total floating-point operations	4.37 x 10^16
Validation speed	6.07 samples each second

10.2 Validation loss during training

Table with columns: Step, Training loss, Validation loss
Step	Training loss	Validation loss
50	1.1866	1.2763
100	1.0245	1.1578
150	0.9912	1.1150
200	0.7534	1.0917
End (step 519)	0.5838 (last step)	1.1404

10.3 How to read these numbers

The validation loss went down from step 50 to step 200. The lowest value that the author recorded is 1.0917 at step 200. The final value is 1.1404, which is a small rise. The gap between the training loss and the validation loss also grew in epoch 3. This can be a sign of a small amount of overfitting. The model can fit the training examples a little too much.

This card lists only the values that the author recorded. The author did not run public benchmarks, such as MATH, GSM8K, or MMLU. Loss is not the same as accuracy. Do not use these numbers as proof of correct answers.

11. Limitations

Size. The model has 4 billion parameters. It makes more errors than a large model.
Wrong numbers. The model can make mistakes in arithmetic, in units, and in long derivations.
Made-up facts. The model can invent exact values, such as register addresses, pin numbers, and part limits. Check each value in the datasheet.
Theory focus. The training data is mostly theory. The model is weaker for long programs and for real hardware settings.
No code tests. The author did not compile or run the code that the model writes.
Language. The model is trained on English text. Other languages can give poor results.
Reasoning. The training data does not teach long reasoning in thinking mode. Expect short and direct answers.
Knowledge. The knowledge of the model comes from the base model and the training data. It can be old.
Safety tuning. The author did not add new safety training. The model keeps the behavior of the base model.
Data bias. The data is a choice of the author. The topics are not complete. The physics data is at high-school level. The MATH-500 data is at competition level.
Overfitting. See Section 10.3.

12. Compute used

Table with columns: Item, Value
Item	Value
Hardware	1 NVIDIA Tesla T4
Training time	About 1 hour 14 minutes
Platform	Google Colab free tier

13. Files in this repository

Table with columns: File, Use
File	Use
`adapter_model.safetensors`	Weights of the LoRA adapter
`adapter_config.json`	Settings of the adapter and the name of the base model
`tokenizer.json`, `tokenizer_config.json`, and related files	Tokenizer and chat template
`README.md`	This model card

14. License

The adapter has the license Apache-2.0. The base model Qwen3-4B has the license Apache-2.0. The datasets MATH-500 and high-school-physics have the license MIT. You must follow the terms of each license. Keep the license notices when you share the model.

15. Citations

15.1 How to cite this model

bibtex

@misc{aegis_mini_1_0_lora_2026,  title        = {Aegis-mini-1.0 (AEGIS-Mini-V1-LoRA): A LoRA Adapter of Qwen3-4B for Electronics, Drones, Robotics, Math, and Physics},  author       = {Xerxes-28},  year         = {2026},  howpublished = {\url{https://huggingface.co/Xerxes-28/AEGIS-Mini-V1-LoRA}},  note         = {Hugging Face model repository}}

15.2 Base model

bibtex

@article{yang2025qwen3,  title   = {Qwen3 Technical Report},  author  = {Yang, An and others},  journal = {arXiv preprint arXiv:2505.09388},  year    = {2025},  url     = {https://arxiv.org/abs/2505.09388}}

15.3 Datasets

bibtex

@inproceedings{hendrycks2021math,  title     = {Measuring Mathematical Problem Solving With the {MATH} Dataset},  author    = {Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob},  booktitle = {Thirty-fifth Conference on Neural Information Processing Systems, Datasets and Benchmarks Track},  year      = {2021},  url       = {https://arxiv.org/abs/2103.03874}} @inproceedings{lightman2024lets,  title     = {Let's Verify Step by Step},  author    = {Lightman, Hunter and Kosaraju, Vineet and Burda, Yura and Edwards, Harri and Baker, Bowen and Lee, Teddy and Leike, Jan and Schulman, John and Sutskever, Ilya and Cobbe, Karl},  booktitle = {The Twelfth International Conference on Learning Representations},  year      = {2024},  url       = {https://arxiv.org/abs/2305.20050}} @misc{h4_math500,  title        = {MATH-500},  author       = {{Hugging Face H4}},  howpublished = {\url{https://huggingface.co/datasets/HuggingFaceH4/MATH-500}},  note         = {Hugging Face dataset. The 500-problem test subset that OpenAI selected in Lightman et al. (2024).}} @misc{mrohith29_highschoolphysics,  title        = {high-school-physics},  author       = {mrohith29},  howpublished = {\url{https://huggingface.co/datasets/mrohith29/high-school-physics}},  note         = {Hugging Face dataset, MIT license}}

15.4 Methods

bibtex

@inproceedings{hu2022lora,  title     = {{LoRA}: Low-Rank Adaptation of Large Language Models},  author    = {Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},  booktitle = {International Conference on Learning Representations},  year      = {2022},  url       = {https://arxiv.org/abs/2106.09685}} @inproceedings{dettmers2023qlora,  title     = {{QLoRA}: Efficient Finetuning of Quantized {LLMs}},  author    = {Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},  booktitle = {Advances in Neural Information Processing Systems},  year      = {2023},  url       = {https://arxiv.org/abs/2305.14314}} @inproceedings{dettmers2022optimizers,  title     = {8-bit Optimizers via Block-wise Quantization},  author    = {Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke},  booktitle = {International Conference on Learning Representations},  year      = {2022},  url       = {https://arxiv.org/abs/2110.02861}} @inproceedings{loshchilov2019adamw,  title     = {Decoupled Weight Decay Regularization},  author    = {Loshchilov, Ilya and Hutter, Frank},  booktitle = {International Conference on Learning Representations},  year      = {2019},  url       = {https://arxiv.org/abs/1711.05101}} @inproceedings{loshchilov2017sgdr,  title     = {{SGDR}: Stochastic Gradient Descent with Warm Restarts},  author    = {Loshchilov, Ilya and Hutter, Frank},  booktitle = {International Conference on Learning Representations},  year      = {2017},  url       = {https://arxiv.org/abs/1608.03983}}

15.5 Software

bibtex

@misc{unsloth,  title        = {Unsloth},  author       = {Han, Daniel and Han, Michael and {Unsloth team}},  year         = {2023},  howpublished = {\url{https://github.com/unslothai/unsloth}}} @misc{vonwerra2020trl,  title        = {{TRL}: Transformer Reinforcement Learning},  author       = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallou{\'e}dec, Quentin},  year         = {2020},  howpublished = {\url{https://github.com/huggingface/trl}}} @inproceedings{wolf2020transformers,  title     = {Transformers: State-of-the-Art Natural Language Processing},  author    = {Wolf, Thomas and others},  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},  pages     = {38--45},  year      = {2020},  url       = {https://aclanthology.org/2020.emnlp-demos.6}} @misc{wandb,  title        = {Experiment Tracking with Weights and Biases},  author       = {Biewald, Lukas},  year         = {2020},  howpublished = {\url{https://www.wandb.com/}}}

16. Contact

Open a discussion in the Community tab of this repository. Hugging Face name of the author: Xerxes-28.

Summary
Model details
Intended use
Safety information
How to use the adapter
Prompt format
Generation settings
Training data
Training procedure
Results
Limitations
Compute used
Files in this repository
License
Citations
Contact

1. Summary

2. Model details

Table with columns: Item, Value
Item	Value
Model name	Aegis-mini-1.0
Repository	`Xerxes-28/AEGIS-Mini-V1-LoRA`
Developer	Xerxes-28
Model type	LoRA adapter (PEFT) for a causal language model
Base model	`unsloth/Qwen3-4B-unsloth-bnb-4bit` (4-bit copy of Qwen3-4B)
Language	English
Task	Text generation (chat)
Training method	Supervised fine-tuning (SFT) with QLoRA
Trainable parameters	33,030,144 (0.81% of 4,055,498,240)
Training context length	2,048 tokens
Training date	2026-10-31
License	Apache-2.0 (same as the base model)
GGUF version	Xerxes-28/AEGIS-Mini-V1-GGUF

3. Intended use

3.1 Use cases

You can use the model for these tasks:

Explain a concept in electronics, for example Ohm's law, filters, or transistor bias.
Explain a concept in drones and robotics, for example PID control or sensor fusion.
Do a step-by-step calculation in math or physics.
Give a first draft of an answer that you then check.
Study and learn. The model is a good partner for practice.

3.2 Uses that are not in scope

Do not use the model for these tasks:

Safety-critical decisions of any kind.
Final design values for a real circuit, battery pack, or aircraft.
Medical, legal, or financial advice.
Exact data from a datasheet, for example register addresses or pin numbers.

4. Safety information

WARNING DO NOT USE THE MODEL OUTPUT AS THE ONLY SOURCE FOR A SAFETY CALCULATION. The model can give a wrong number with a confident tone. A wrong number can cause injury or fire.

WARNING DO NOT TOUCH MAINS VOLTAGE OR HIGH-VOLTAGE CIRCUITS BECAUSE THE MODEL TOLD YOU IT IS SAFE. Electrical shock can cause injury or death. Ask a qualified person.

CAUTION Make sure that you check each value in the datasheet of the part. A wrong value can cause damage to the part, the battery, or the drone.

CAUTION Do not fly a drone with a setting that the model gave you before you test it in a safe place. A wrong control setting can cause a crash.

5. How to use the adapter

You need the adapter and the base model. There are three ways to load them.

5.1 Method A: Unsloth (fastest)

Unsloth loads the base model and the adapter together.

python

from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained(    model_name="Xerxes-28/AEGIS-Mini-V1-LoRA",    max_seq_length=2048,    load_in_4bit=True,)FastLanguageModel.for_inference(model)

from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained(    model_name="Xerxes-28/AEGIS-Mini-V1-LoRA",    max_seq_length=2048,    load_in_4bit=True,)FastLanguageModel.for_inference(model)

Do not wrap this model again with PeftModel. Unsloth reads adapter_config.json and loads the adapter by itself.

5.2 Method B: Transformers and PEFT

Use this method when you do not want Unsloth. You need transformers, peft, accelerate, and bitsandbytes.

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_id = "unsloth/Qwen3-4B-unsloth-bnb-4bit"   # same base that the author used for trainingadapter_id = "Xerxes-28/AEGIS-Mini-V1-LoRA" tokenizer = AutoTokenizer.from_pretrained(adapter_id)model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")model = PeftModel.from_pretrained(model, adapter_id)model.eval()

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_id = "unsloth/Qwen3-4B-unsloth-bnb-4bit"   # same base that the author used for trainingadapter_id = "Xerxes-28/AEGIS-Mini-V1-LoRA" tokenizer = AutoTokenizer.from_pretrained(adapter_id)model = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")model = PeftModel.from_pretrained(model, adapter_id)model.eval()

5.3 Run one answer

python

SYSTEM_PROMPT = (    "You are Aegis-mini-1.0, a compact AI assistant for electronics, drones, robotics, "    "math and physics. You are strongest at theory and calculations. Be concise: state the "    "key formula, show the steps with units, then give the result. If you are unsure of an "    "exact value (register address, pin number, part spec), say so and tell the user to "    "check the datasheet instead of guessing. For topics outside your focus, say so briefly.") messages = [    {"role": "system", "content": SYSTEM_PROMPT},    {"role": "user", "content": "A 5 V supply drives a red LED (2.0 V, 10 mA) through a resistor. Find the resistor value."},] inputs = tokenizer.apply_chat_template(    messages,    add_generation_prompt=True,    return_tensors="pt",    return_dict=True,).to(model.device) with torch.no_grad():    output = model.generate(        **inputs,        max_new_tokens=512,        do_sample=True,        temperature=0.3,        top_p=0.8,        top_k=20,        min_p=0.0,        repetition_penalty=1.05,    ) new_tokens = output[0][inputs["input_ids"].shape[1]:]print(tokenizer.decode(new_tokens, skip_special_tokens=True))

SYSTEM_PROMPT = (    "You are Aegis-mini-1.0, a compact AI assistant for electronics, drones, robotics, "    "math and physics. You are strongest at theory and calculations. Be concise: state the "    "key formula, show the steps with units, then give the result. If you are unsure of an "    "exact value (register address, pin number, part spec), say so and tell the user to "    "check the datasheet instead of guessing. For topics outside your focus, say so briefly.") messages = [    {"role": "system", "content": SYSTEM_PROMPT},    {"role": "user", "content": "A 5 V supply drives a red LED (2.0 V, 10 mA) through a resistor. Find the resistor value."},] inputs = tokenizer.apply_chat_template(    messages,    add_generation_prompt=True,    return_tensors="pt",    return_dict=True,).to(model.device) with torch.no_grad():    output = model.generate(        **inputs,        max_new_tokens=512,        do_sample=True,        temperature=0.3,        top_p=0.8,        top_k=20,        min_p=0.0,        repetition_penalty=1.05,    ) new_tokens = output[0][inputs["input_ids"].shape[1]:]print(tokenizer.decode(new_tokens, skip_special_tokens=True))

5.4 Merge the adapter into the base model (optional)

Merge the adapter when you want one set of weights, for example to make a GGUF file.

python

from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained(    model_name="Xerxes-28/AEGIS-Mini-V1-LoRA",    max_seq_length=2048,    load_in_4bit=True,)model.save_pretrained_merged("aegis-mini-merged", tokenizer, save_method="merged_16bit")

from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained(    model_name="Xerxes-28/AEGIS-Mini-V1-LoRA",    max_seq_length=2048,    load_in_4bit=True,)model.save_pretrained_merged("aegis-mini-merged", tokenizer, save_method="merged_16bit")

The base weights are in 4-bit form during training. The merge step changes them back to 16-bit numbers. This step can add small rounding errors.

6. Prompt format

The model uses the chat template of Qwen3. The template uses the ChatML structure.

text

<|im_start|>system{system prompt}<|im_end|><|im_start|>user{question}<|im_end|><|im_start|>assistant

<|im_start|>system{system prompt}<|im_end|><|im_start|>user{question}<|im_end|><|im_start|>assistant

Call tokenizer.apply_chat_template to build this text. Do not type the tags by hand.

6.1 Recommended system prompt

text

You are Aegis-mini-1.0, a compact AI assistant for electronics, drones, robotics, math and physics. You are strongest at theory and calculations. Be concise: state the key formula, show the steps with units, then give the result. If you are unsure of an exact value (register address, pin number, part spec), say so and tell the user to check the datasheet instead of guessing. For topics outside your focus, say so briefly.

You are Aegis-mini-1.0, a compact AI assistant for electronics, drones, robotics, math and physics. You are strongest at theory and calculations. Be concise: state the key formula, show the steps with units, then give the result. If you are unsure of an exact value (register address, pin number, part spec), say so and tell the user to check the datasheet instead of guessing. For topics outside your focus, say so briefly.

Keep the system prompt short. A long system prompt uses context and makes a phone slow.

7. Generation settings

Table with columns: Setting, Recommended value, Remark
Setting	Recommended value	Remark
Temperature	0.3 to 0.7	Use 0.3 for calculations and code. Use 0.7 for explanations.
Top-k	20
Top-p	0.8
Min-p	0
Repetition penalty	1.0 to 1.05	Do not go above 1.1.
Maximum new tokens	512 to 1024	Use a higher value for long derivations.
Context length	2,048 to 4,096	The author trained with 2,048 tokens.

CAUTION Do not set a no-repeat n-gram size. Code and equations repeat groups of tokens on purpose. This setting breaks them.

8. Training data

8.1 Sources

Table with columns: Source, Lines used, License, Remark
Source	Lines used	License	Remark
Dataset of the author	Not listed in this card	Not published	Mostly theory in electronics, drones, robotics, math, and physics.
HuggingFaceH4/MATH-500	500	MIT	500 problems and worked solutions from the MATH benchmark.
mrohith29/high-school-physics	400	MIT	High-school physics questions and answers.

The author did not publish the dataset of the author. The author can publish it later.

8.2 Data preparation

The author did these steps in order:

Convert each row into chat messages with a user role and an assistant role.
Add a system prompt to each example when a system prompt was set.
Apply the chat template of Qwen3 to make one text for each example.
Count the tokens of each example.
Remove each example that has more than 2,048 tokens.
Shuffle the data with seed 42.
Hold out 100 examples for validation.

The result is 2,765 training examples and 100 validation examples.

8.3 Loss mask

9. Training procedure

9.1 Hardware and software

Table with columns: Item, Value
Item	Value
GPU	NVIDIA Tesla T4 (14.6 GB), Google Colab free tier
Number of GPUs	1
Python	3.13
Main libraries	Unsloth, TRL, Transformers, PEFT, bitsandbytes, PyTorch
Experiment tracking	Weights & Biases 0.28.1
Library versions	The author did not record the exact versions.

9.2 LoRA settings

Table with columns: Setting, Value
Setting	Value
Rank (r)	16
Alpha	32
Dropout	0
Bias	none
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Gradient checkpointing	Unsloth method
Random state	3407

9.3 Training settings

Table with columns: Setting, Value
Setting	Value
Epochs	3
Total optimizer steps	519
Batch size on each device	2
Gradient accumulation steps	8
Effective batch size	16
Peak learning rate	2e-4
Learning rate schedule	Cosine, to near zero at the end
Warmup	20 steps
Weight decay	0.01
Optimizer	AdamW 8-bit
Precision	Mixed precision, fp16 (the T4 does not support bf16)
Maximum sequence length	2,048 tokens
Seed	3407
Validation	Every 50 steps
Checkpoint	Every 100 steps (the last 2 are kept)

10. Results

10.1 Final numbers

Table with columns: Metric, Value
Metric	Value
Final validation loss	1.1404
Perplexity from the final validation loss	about 3.13
Training loss (average of the whole run)	0.8454
Training loss (last logged step)	0.5838
Final gradient norm	0.848
Final learning rate	1.98e-7
Epochs finished	3
Steps finished	519
Training time	4,435 seconds (about 1 hour 14 minutes)
Training speed	1.87 samples each second, 0.117 steps each second
Total floating-point operations	4.37 x 10^16
Validation speed	6.07 samples each second

10.2 Validation loss during training

Table with columns: Step, Training loss, Validation loss
Step	Training loss	Validation loss
50	1.1866	1.2763
100	1.0245	1.1578
150	0.9912	1.1150
200	0.7534	1.0917
End (step 519)	0.5838 (last step)	1.1404

10.3 How to read these numbers

11. Limitations

Size. The model has 4 billion parameters. It makes more errors than a large model.
Wrong numbers. The model can make mistakes in arithmetic, in units, and in long derivations.
Made-up facts. The model can invent exact values, such as register addresses, pin numbers, and part limits. Check each value in the datasheet.
Theory focus. The training data is mostly theory. The model is weaker for long programs and for real hardware settings.
No code tests. The author did not compile or run the code that the model writes.
Language. The model is trained on English text. Other languages can give poor results.
Reasoning. The training data does not teach long reasoning in thinking mode. Expect short and direct answers.
Knowledge. The knowledge of the model comes from the base model and the training data. It can be old.
Safety tuning. The author did not add new safety training. The model keeps the behavior of the base model.
Data bias. The data is a choice of the author. The topics are not complete. The physics data is at high-school level. The MATH-500 data is at competition level.
Overfitting. See Section 10.3.

12. Compute used

Table with columns: Item, Value
Item	Value
Hardware	1 NVIDIA Tesla T4
Training time	About 1 hour 14 minutes
Platform	Google Colab free tier

13. Files in this repository

Table with columns: File, Use
File	Use
`adapter_model.safetensors`	Weights of the LoRA adapter
`adapter_config.json`	Settings of the adapter and the name of the base model
`tokenizer.json`, `tokenizer_config.json`, and related files	Tokenizer and chat template
`README.md`	This model card

14. License

15. Citations

15.1 How to cite this model

bibtex

@misc{aegis_mini_1_0_lora_2026,  title        = {Aegis-mini-1.0 (AEGIS-Mini-V1-LoRA): A LoRA Adapter of Qwen3-4B for Electronics, Drones, Robotics, Math, and Physics},  author       = {Xerxes-28},  year         = {2026},  howpublished = {\url{https://huggingface.co/Xerxes-28/AEGIS-Mini-V1-LoRA}},  note         = {Hugging Face model repository}}

@misc{aegis_mini_1_0_lora_2026,  title        = {Aegis-mini-1.0 (AEGIS-Mini-V1-LoRA): A LoRA Adapter of Qwen3-4B for Electronics, Drones, Robotics, Math, and Physics},  author       = {Xerxes-28},  year         = {2026},  howpublished = {\url{https://huggingface.co/Xerxes-28/AEGIS-Mini-V1-LoRA}},  note         = {Hugging Face model repository}}

15.2 Base model

bibtex

@article{yang2025qwen3,  title   = {Qwen3 Technical Report},  author  = {Yang, An and others},  journal = {arXiv preprint arXiv:2505.09388},  year    = {2025},  url     = {https://arxiv.org/abs/2505.09388}}

@article{yang2025qwen3,  title   = {Qwen3 Technical Report},  author  = {Yang, An and others},  journal = {arXiv preprint arXiv:2505.09388},  year    = {2025},  url     = {https://arxiv.org/abs/2505.09388}}

15.3 Datasets

bibtex

@inproceedings{hendrycks2021math,  title     = {Measuring Mathematical Problem Solving With the {MATH} Dataset},  author    = {Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob},  booktitle = {Thirty-fifth Conference on Neural Information Processing Systems, Datasets and Benchmarks Track},  year      = {2021},  url       = {https://arxiv.org/abs/2103.03874}} @inproceedings{lightman2024lets,  title     = {Let's Verify Step by Step},  author    = {Lightman, Hunter and Kosaraju, Vineet and Burda, Yura and Edwards, Harri and Baker, Bowen and Lee, Teddy and Leike, Jan and Schulman, John and Sutskever, Ilya and Cobbe, Karl},  booktitle = {The Twelfth International Conference on Learning Representations},  year      = {2024},  url       = {https://arxiv.org/abs/2305.20050}} @misc{h4_math500,  title        = {MATH-500},  author       = {{Hugging Face H4}},  howpublished = {\url{https://huggingface.co/datasets/HuggingFaceH4/MATH-500}},  note         = {Hugging Face dataset. The 500-problem test subset that OpenAI selected in Lightman et al. (2024).}} @misc{mrohith29_highschoolphysics,  title        = {high-school-physics},  author       = {mrohith29},  howpublished = {\url{https://huggingface.co/datasets/mrohith29/high-school-physics}},  note         = {Hugging Face dataset, MIT license}}

@inproceedings{hendrycks2021math,  title     = {Measuring Mathematical Problem Solving With the {MATH} Dataset},  author    = {Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob},  booktitle = {Thirty-fifth Conference on Neural Information Processing Systems, Datasets and Benchmarks Track},  year      = {2021},  url       = {https://arxiv.org/abs/2103.03874}} @inproceedings{lightman2024lets,  title     = {Let's Verify Step by Step},  author    = {Lightman, Hunter and Kosaraju, Vineet and Burda, Yura and Edwards, Harri and Baker, Bowen and Lee, Teddy and Leike, Jan and Schulman, John and Sutskever, Ilya and Cobbe, Karl},  booktitle = {The Twelfth International Conference on Learning Representations},  year      = {2024},  url       = {https://arxiv.org/abs/2305.20050}} @misc{h4_math500,  title        = {MATH-500},  author       = {{Hugging Face H4}},  howpublished = {\url{https://huggingface.co/datasets/HuggingFaceH4/MATH-500}},  note         = {Hugging Face dataset. The 500-problem test subset that OpenAI selected in Lightman et al. (2024).}} @misc{mrohith29_highschoolphysics,  title        = {high-school-physics},  author       = {mrohith29},  howpublished = {\url{https://huggingface.co/datasets/mrohith29/high-school-physics}},  note         = {Hugging Face dataset, MIT license}}

15.4 Methods

bibtex

@inproceedings{hu2022lora,  title     = {{LoRA}: Low-Rank Adaptation of Large Language Models},  author    = {Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},  booktitle = {International Conference on Learning Representations},  year      = {2022},  url       = {https://arxiv.org/abs/2106.09685}} @inproceedings{dettmers2023qlora,  title     = {{QLoRA}: Efficient Finetuning of Quantized {LLMs}},  author    = {Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},  booktitle = {Advances in Neural Information Processing Systems},  year      = {2023},  url       = {https://arxiv.org/abs/2305.14314}} @inproceedings{dettmers2022optimizers,  title     = {8-bit Optimizers via Block-wise Quantization},  author    = {Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke},  booktitle = {International Conference on Learning Representations},  year      = {2022},  url       = {https://arxiv.org/abs/2110.02861}} @inproceedings{loshchilov2019adamw,  title     = {Decoupled Weight Decay Regularization},  author    = {Loshchilov, Ilya and Hutter, Frank},  booktitle = {International Conference on Learning Representations},  year      = {2019},  url       = {https://arxiv.org/abs/1711.05101}} @inproceedings{loshchilov2017sgdr,  title     = {{SGDR}: Stochastic Gradient Descent with Warm Restarts},  author    = {Loshchilov, Ilya and Hutter, Frank},  booktitle = {International Conference on Learning Representations},  year      = {2017},  url       = {https://arxiv.org/abs/1608.03983}}

@inproceedings{hu2022lora,  title     = {{LoRA}: Low-Rank Adaptation of Large Language Models},  author    = {Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},  booktitle = {International Conference on Learning Representations},  year      = {2022},  url       = {https://arxiv.org/abs/2106.09685}} @inproceedings{dettmers2023qlora,  title     = {{QLoRA}: Efficient Finetuning of Quantized {LLMs}},  author    = {Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},  booktitle = {Advances in Neural Information Processing Systems},  year      = {2023},  url       = {https://arxiv.org/abs/2305.14314}} @inproceedings{dettmers2022optimizers,  title     = {8-bit Optimizers via Block-wise Quantization},  author    = {Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke},  booktitle = {International Conference on Learning Representations},  year      = {2022},  url       = {https://arxiv.org/abs/2110.02861}} @inproceedings{loshchilov2019adamw,  title     = {Decoupled Weight Decay Regularization},  author    = {Loshchilov, Ilya and Hutter, Frank},  booktitle = {International Conference on Learning Representations},  year      = {2019},  url       = {https://arxiv.org/abs/1711.05101}} @inproceedings{loshchilov2017sgdr,  title     = {{SGDR}: Stochastic Gradient Descent with Warm Restarts},  author    = {Loshchilov, Ilya and Hutter, Frank},  booktitle = {International Conference on Learning Representations},  year      = {2017},  url       = {https://arxiv.org/abs/1608.03983}}

15.5 Software

bibtex

@misc{unsloth,  title        = {Unsloth},  author       = {Han, Daniel and Han, Michael and {Unsloth team}},  year         = {2023},  howpublished = {\url{https://github.com/unslothai/unsloth}}} @misc{vonwerra2020trl,  title        = {{TRL}: Transformer Reinforcement Learning},  author       = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallou{\'e}dec, Quentin},  year         = {2020},  howpublished = {\url{https://github.com/huggingface/trl}}} @inproceedings{wolf2020transformers,  title     = {Transformers: State-of-the-Art Natural Language Processing},  author    = {Wolf, Thomas and others},  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},  pages     = {38--45},  year      = {2020},  url       = {https://aclanthology.org/2020.emnlp-demos.6}} @misc{wandb,  title        = {Experiment Tracking with Weights and Biases},  author       = {Biewald, Lukas},  year         = {2020},  howpublished = {\url{https://www.wandb.com/}}}

@misc{unsloth,  title        = {Unsloth},  author       = {Han, Daniel and Han, Michael and {Unsloth team}},  year         = {2023},  howpublished = {\url{https://github.com/unslothai/unsloth}}} @misc{vonwerra2020trl,  title        = {{TRL}: Transformer Reinforcement Learning},  author       = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallou{\'e}dec, Quentin},  year         = {2020},  howpublished = {\url{https://github.com/huggingface/trl}}} @inproceedings{wolf2020transformers,  title     = {Transformers: State-of-the-Art Natural Language Processing},  author    = {Wolf, Thomas and others},  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},  pages     = {38--45},  year      = {2020},  url       = {https://aclanthology.org/2020.emnlp-demos.6}} @misc{wandb,  title        = {Experiment Tracking with Weights and Biases},  author       = {Biewald, Lukas},  year         = {2020},  howpublished = {\url{https://www.wandb.com/}}}

16. Contact

Open a discussion in the Community tab of this repository. Hugging Face name of the author: Xerxes-28.

AEGIS-Mini-V1-LoRA

README

Table of contents

1. Summary

2. Model details

3. Intended use

3.1 Use cases

3.2 Uses that are not in scope

4. Safety information

5. How to use the adapter

5.1 Method A: Unsloth (fastest)

5.2 Method B: Transformers and PEFT

5.3 Run one answer

5.4 Merge the adapter into the base model (optional)

6. Prompt format

6.1 Recommended system prompt

7. Generation settings

8. Training data

8.1 Sources

8.2 Data preparation

8.3 Loss mask

9. Training procedure

9.1 Hardware and software

9.2 LoRA settings

9.3 Training settings

10. Results

10.1 Final numbers

10.2 Validation loss during training

10.3 How to read these numbers

11. Limitations

12. Compute used

13. Files in this repository

14. License

15. Citations

15.1 How to cite this model

15.2 Base model

15.3 Datasets

15.4 Methods

15.5 Software

16. Contact

Explore FriendliAI today

README

Table of contents

1. Summary

2. Model details

3. Intended use

3.1 Use cases

3.2 Uses that are not in scope

4. Safety information

5. How to use the adapter

5.1 Method A: Unsloth (fastest)

5.2 Method B: Transformers and PEFT

5.3 Run one answer

5.4 Merge the adapter into the base model (optional)

6. Prompt format

6.1 Recommended system prompt

7. Generation settings

8. Training data

8.1 Sources

8.2 Data preparation

8.3 Loss mask

9. Training procedure

9.1 Hardware and software

9.2 LoRA settings

9.3 Training settings

10. Results

10.1 Final numbers

10.2 Validation loss during training

10.3 How to read these numbers

11. Limitations

12. Compute used

13. Files in this repository

14. License

15. Citations

15.1 How to cite this model

15.2 Base model

15.3 Datasets

15.4 Methods

15.5 Software

16. Contact