Base Model
- Base Model: mistralai/Mistral-7B-Instruct-v0.3
- Fine-tuning Method: LoRA (PEFT)
- Merged: Yes
- Framework: Transformers + PEFT + Unsloth
- Language: English
- License: Apache-2.0
Intended Use
The model is intended for generating English MCQs for educational purposes.
Typical tasks include:
- Grammar Questions
- Vocabulary Questions
- Reading-based MCQs
- Curriculum-aligned English assessments
- Synthetic educational data generation
Example Prompt
Generate 5 official Egyptian Ministry of Education English multiple-choice questions.
Unit: 10
Topic:
At the Airport
Grammar Focus:
Past Perfect / Past Perfect Continuous
Return only valid JSON.
Example Output
{
"questions": [
{
"statement": "...",
"correct_answer": "...",
"plausible_distractors": [
"...",
"...",
"..."
],
"explanation": "..."
}
]
}
Training
Dataset
The model was fine-tuned on a synthetic instruction dataset containing English examination questions inspired by the Egyptian Ministry of Education curriculum.
The dataset includes:
- Grammar
- Vocabulary
- Reading
- Functional Language
- Writing
- Multiple curriculum units
All samples are formatted as multi-turn chat conversations.
Fine-tuning
- LoRA Rank: 16
- LoRA Alpha: 32
- LoRA Dropout: 0.05
- Learning Rate: 1e-4
- Optimizer: AdamW 8-bit
- Scheduler: Cosine
- Max Sequence Length: 2048
- Precision: FP16
- Quantized Base Model: 4-bit (BitsAndBytes NF4)
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "tokhey/egyptian-mcq-generator-mistral-7b-synthetic"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
)
prompt = "Generate 5 grammar questions about Past Perfect."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Limitations
Although the model has been fine-tuned for educational assessment generation, it may occasionally:
- Mix different grammar topics.
- Produce questions outside the requested curriculum unit.
- Generate explanations that are generic.
- Produce invalid JSON in rare cases.
Human review is recommended before using generated questions in official educational settings.
Evaluation
The model has been qualitatively evaluated by comparing its outputs with the original Mistral-7B-Instruct-v0.3 base model.
Fine-tuning improves:
- Instruction following
- JSON formatting
- Educational question generation
- Curriculum-aware responses
Framework Versions
- Transformers
- PEFT 0.19.1
- TRL
- Unsloth
- BitsAndBytes
Author
Ahmed Eltokhy
GitHub:
https://github.com/ahmdeltoky03
LinkedIn:
https://www.linkedin.com/in/ahmed-eltokhy-7b7a073ba