Model Details
This repository contains a math-focused PEFT LoRA adapter based on [openbmb/MiniCPM5-1B](https://huggingface.co/
openbmb/MiniCPM5-1B).
- Base model:
openbmb/MiniCPM5-1B
- Training mode:
lora
- Published artifact:
adapter
- Primary task: grade-school math reasoning
- Primary eval: GSM8K
- Guard eval: PIQA
- LoRA rank:
32
- LoRA alpha:
64
- NEFTune alpha:
5.0
- Max steps:
150
Intended Use
This model is intended for research and experimentation with math reasoning fine-tuning. It is not intended for high-stakes educational, financial, legal, medical, or safety-critical
decisions.
Training Data
The adapter was trained on math reasoning data, including MetaMathQA and Hendrycks MATH-style algebra/geometry examples, depending on the notebook configuration used for this run.
Training Procedure
The run used lora fine-tuning with PEFT/LoRA-style adaptation where applicable. Training configuration and adapter metadata are included in adapter_enhancement_metadata.json.
Evaluation
Before/after evaluation was run with the same base model and PEFT disable_adapter() for the baseline.
Table with columns: Task, Role, N, Before, After, Delta| Task | Role | N | Before | After | Delta |
|---|
| GSM8K | primary | 30 | 0.133 | 0.200 | +0.067 |
| PIQA | guard | 150 | 0.620 | 0.647 | +0.027 |
Gate status: PASS.
How to Use
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = "openbmb/MiniCPM5-1B"
adapter_id = "MSGEncrypted/minicpm5-1b-math-lora"
tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
base_model,
torch_dtype="auto",
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, adapter_id)
prompt = "Solve: If there are 12 apples and 5 are eaten, how many remain?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Limitations
The model may produce incorrect reasoning, hallucinated steps, or invalid arithmetic. Evaluation results from small notebook samples should be treated as smoke tests, not full
benchmark claims.
Artifacts
eval_results.json: captured GSM8K/PIQA notebook results
adapter_enhancement_metadata.json: training and publish metadata
- Adapter/model files produced by the notebook save step
Framework Versions