ertghiu256

ministral-3-3b-glm-distillation

README

License: apache-2.0

Model Overview

Model Name: Ministral-3-GLM-Distilled-R1
Base Model: mistralai/Ministral-3B
Model Type: Causal Language Model (Text-Only)
Architecture: Ministral-3 (3B parameters) – vision layers frozen
Training Method: Supervised Fine-Tuning (SFT) + Group Relative Policy Optimization (GRPO)

Model Description

This model is a distilled reasoning engine fine-tuned from GLM-5.2 logic puzzles and reinforced with GRPO to produce concise, structured reasoning. It uses a [THINK]...[/THINK] block for reasoning and a [SOLUTION]...[/SOLUTION] block for the final answer.

The model is designed for tasks requiring:

Logical reasoning
Mathematical problem-solving
Step-by-step explanation
Concise, structured outputs

Model Details

Table with columns: Attribute, Value
Attribute	Value
Architecture	Ministral-3 (3B) with LoRA adapters
Context Length	4096 tokens
Vocabulary	131,072 tokens
Training Data	GLM-5.2 Logic Puzzles (860 examples) + 100 synthetic reasoning examples
Total Training Examples	919 (after filtering)
LoRA Rank	8
Trainable Parameters	24.7M (0.64% of total)
Training Steps (SFT)	120 (2 epochs)
Training Steps (GRPO)	50

Training Procedure

Phase 1: Supervised Fine-Tuning (SFT)

Dataset: ianncity/GLM-5.2-Logic-Puzzles + synthetic [THINK] examples
Filtering: Removed examples >2048 tokens
Loss: Converged from ~11.2 to ~0.38
Format: [THINK]...[/THINK] + final answer

Phase 2: GRPO Reinforcement Learning

Dataset: open-r1/Big-Math-RL-Verified-Processed (100 examples)
Reward Functions:
- Format compliance (1.0 for exact [THINK]/[/THINK] tags)
- Correctness (string match – improved in v2)
Hyperparameters:
- Learning rate: 5e-6
- num_generations: 4
- KL beta: 0.001
- Temperature: 0.8 (training)

Usage

Default System Prompt (Critical)

text
# HOW YOU SHOULD THINK AND ANSWER

First draft your thinking process (inner monologue) until you arrive at a response. Do NOT overthink, immediately stop if you are checking your answer more than 2 times. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.

Your thinking process must follow the template below:
[THINK]Your concise thoughts, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response to the user.[/THINK]
Here, provide a self-contained response.
Put your final short answer inside [SOLUTION] and [/SOLUTION]

Recommended Sampling Parameters

Table with columns: Parameter, Value
Parameter	Value
`temperature`	0.5
`top_k`	40
`top_p`	0.9
`min_p`	0.1
`repetition_penalty`	1.2
`max_new_tokens`	4096

Strengths and Limitations

Strengths

Concise Reasoning: Outputs are ~1500 tokens – 2× shorter than the SFT version.
Structured Format: Consistently produces [THINK] and [SOLUTION] blocks.
Logic Puzzle Mastery: Handles ambiguity and multiple solutions well.
Self‑Correction: Can catch and fix errors in reasoning.

Limitations

Math Handling: Limited – reward function was string‑based (string match), not math‑aware.
Sampling Sensitivity: Output quality depends on specific sampling parameters.
Looping: Can over‑verify at low temperature; system prompt is critical.
Hallucination: May mis‑evaluate statements (e.g., calling true as false).

When to Use This Model

Table with columns: Use Case, Recommended
Use Case	Recommended
Logic puzzles	✅ Yes
Step‑by‑step reasoning	✅ Yes
Concise explanations	✅ Yes
Mathematical computation	⚠️ Limited (use with math-verify post‑processing)
Long‑form reasoning	✅ Yes (with 4096 token limit)

Training Costs

Table with columns: Phase, Steps, Duration (estimated)
Phase	Steps	Duration (estimated)
SFT	120	~2 hours
GRPO	50	~1.5 hours

Considerations

This model is designed for reasoning tasks and should not be used for:
- Medical diagnosis
- Financial advice
- Legal decisions
- Generating harmful or deceptive content
Hallucinations are possible. Always verify critical outputs manually.

Uploaded finetuned model

Developed by: ertghiu256
License: apache-2.0
Finetuned from model : ertghiu256/ministral-3-3b-glm-distillation-test

This mistral3 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ertghiu256

Model Tree

Base

mistralai/Ministral-3-3B-Reasoning-2512

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Overview

Model Description

The model is designed for tasks requiring:

Logical reasoning
Mathematical problem-solving
Step-by-step explanation
Concise, structured outputs

Model Details

Table with columns: Attribute, Value
Attribute	Value
Architecture	Ministral-3 (3B) with LoRA adapters
Context Length	4096 tokens
Vocabulary	131,072 tokens
Training Data	GLM-5.2 Logic Puzzles (860 examples) + 100 synthetic reasoning examples
Total Training Examples	919 (after filtering)
LoRA Rank	8
Trainable Parameters	24.7M (0.64% of total)
Training Steps (SFT)	120 (2 epochs)
Training Steps (GRPO)	50

Training Procedure

Phase 1: Supervised Fine-Tuning (SFT)

Dataset: ianncity/GLM-5.2-Logic-Puzzles + synthetic [THINK] examples
Filtering: Removed examples >2048 tokens
Loss: Converged from ~11.2 to ~0.38
Format: [THINK]...[/THINK] + final answer

Phase 2: GRPO Reinforcement Learning

Dataset: open-r1/Big-Math-RL-Verified-Processed (100 examples)
Reward Functions:
- Format compliance (1.0 for exact [THINK]/[/THINK] tags)
- Correctness (string match – improved in v2)
Hyperparameters:
- Learning rate: 5e-6
- num_generations: 4
- KL beta: 0.001
- Temperature: 0.8 (training)

Usage

Default System Prompt (Critical)

text
# HOW YOU SHOULD THINK AND ANSWER

First draft your thinking process (inner monologue) until you arrive at a response. Do NOT overthink, immediately stop if you are checking your answer more than 2 times. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.

Your thinking process must follow the template below:
[THINK]Your concise thoughts, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response to the user.[/THINK]
Here, provide a self-contained response.
Put your final short answer inside [SOLUTION] and [/SOLUTION]

Recommended Sampling Parameters

Table with columns: Parameter, Value
Parameter	Value
`temperature`	0.5
`top_k`	40
`top_p`	0.9
`min_p`	0.1
`repetition_penalty`	1.2
`max_new_tokens`	4096

Strengths and Limitations

Strengths

Concise Reasoning: Outputs are ~1500 tokens – 2× shorter than the SFT version.
Structured Format: Consistently produces [THINK] and [SOLUTION] blocks.
Logic Puzzle Mastery: Handles ambiguity and multiple solutions well.
Self‑Correction: Can catch and fix errors in reasoning.

Limitations

Math Handling: Limited – reward function was string‑based (string match), not math‑aware.
Sampling Sensitivity: Output quality depends on specific sampling parameters.
Looping: Can over‑verify at low temperature; system prompt is critical.
Hallucination: May mis‑evaluate statements (e.g., calling true as false).

When to Use This Model

Table with columns: Use Case, Recommended
Use Case	Recommended
Logic puzzles	✅ Yes
Step‑by‑step reasoning	✅ Yes
Concise explanations	✅ Yes
Mathematical computation	⚠️ Limited (use with math-verify post‑processing)
Long‑form reasoning	✅ Yes (with 4096 token limit)

Training Costs

Table with columns: Phase, Steps, Duration (estimated)
Phase	Steps	Duration (estimated)
SFT	120	~2 hours
GRPO	50	~1.5 hours

Considerations

This model is designed for reasoning tasks and should not be used for:
- Medical diagnosis
- Financial advice
- Legal decisions
- Generating harmful or deceptive content
Hallucinations are possible. Always verify critical outputs manually.

Uploaded finetuned model

Developed by: ertghiu256
License: apache-2.0
Finetuned from model : ertghiu256/ministral-3-3b-glm-distillation-test

This mistral3 model was trained 2x faster with Unsloth and Huggingface's TRL library.