Model Details
Model Description
Qwen2.5-3B Korean Petition 100K is a supervised fine-tuned language model based on Qwen2.5-3B. The model was trained on approximately 100,000 Korean public petition samples to improve its ability to understand and generate Korean formal writing, public opinions, and petition-style text.
The model is intended for research purposes and experimentation with Korean language generation and instruction-following tasks.
- Developed by: Asel Nurlanbek kyzy
- Funded by: Personal research / Korea University of Technology and Education (KOREATECH)
- Shared by: aselbaekki
- Model type: Causal Language Model
- Language(s): Korean
- License: Apache 2.0
- Finetuned from model: Qwen/Qwen2.5-3B
Model Sources
Uses
Direct Use
The model can be used for
- Korean text generation
- Korean instruction following
- Petition generation
- Text completion
- Korean NLP research
- Educational purposes
Downstream Use
Possible downstream applications include
- Chatbots
- Summarization
- Question Answering
- Writing assistance
- Public opinion analysis
- Korean document generation
Out-of-Scope Use
This model should not be used for
- Medical advice
- Legal advice
- Financial decision making
- Safety-critical applications
- Generating misinformation or harmful content
Bias, Risks, and Limitations
The model inherits limitations from the original Qwen2.5-3B model and the petition dataset used during fine-tuning.
Potential limitations include:
- Biases present in public petition data
- Hallucinated information
- Limited factual reliability
- Performance degradation outside Korean language tasks
Users should verify important outputs before use.
How to Get Started
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "aselbaekki/Qwen2.5-3B-korean-petition-100k"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
prompt = "대한민국 교육 정책에 대해 설명해주세요."
messages = [
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training Details
Training Data
The model was fine-tuned on approximately 100,000 Korean petition samples.
The dataset consists primarily of Korean public petitions containing formal writing and citizen opinions.
Training Procedure
Preprocessing
- Tokenization using the original Qwen2.5 tokenizer
- Standard chat template formatting
- Sequence truncation and padding
Training Hyperparameters
Table with columns: Parameter, Value| Parameter | Value |
|---|
| Base Model | Qwen2.5-3B |
| Epochs | 1 |
| Learning Rate | 5e-5 |
| Batch Size | 4 |
| Gradient Accumulation | 4 |
| Scheduler | Cosine |
| Warmup Ratio | 0.03 |
| Precision | BF16 |
| Optimizer | AdamW |
Evaluation
The model was qualitatively evaluated on Korean instruction-following and text generation tasks.
Future work will include standardized benchmarks for
- Ko-MMLU
- KMMLU
- Korean instruction-following evaluation
- Human evaluation
Technical Specifications
Model Architecture
- Architecture: Qwen2ForCausalLM
- Parameters: 3 Billion
- Context Length: 32K tokens (base model capability)
- Framework: Hugging Face Transformers
Compute Infrastructure
Hardware
- NVIDIA GPU
- BF16 mixed precision
Software
- Python
- PyTorch
- Transformers
- Accelerate
- Hugging Face Trainer
Citation
If you use this model in your research, please cite both the original Qwen2.5 paper and this repository.
@misc{qwen25-korean-petition-100k,
author = {Asel Nurlanbek kyzy},
title = {Qwen2.5-3B Korean Petition 100K},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/aselbaekki/Qwen2.5-3B-korean-petition-100k}}
}
For questions or feedback, please open an issue on the Hugging Face repository or contact the repository owner.
Hugging Face: https://huggingface.co/aselbaekki