Base model
Qwen/Qwen3.5-2B, released under Apache-2.0. It is a composite text and vision model. Only the text transformer was trained here. The vision components are carried over unchanged and were not tested.
Method
- LoRA supervised fine-tuning (rank 128) on a Danish instruction mix of about 32,000 examples: synthetic instruction pairs generated with the DeepSeek API from modern Danish source text, a set of examples written in EuroEval answer formats (single-word sentiment labels, yes/no grammaticality, multiple-choice letters), the skolegpt-instruct dataset, and roughly 12% English data to limit forgetting.
- The trained adapter was folded into the base weights at strength 0.3, following the WiSE-FT idea (W = W_base + 0.3 x delta). Merging at full strength improved generation tasks but hurt the constrained-choice tasks; a partial merge kept most of the Danish gains while limiting those regressions. Strength 0.3 gave the best average on the validation split, and that choice was then checked once on the test split.
Evaluation
EuroEval, Danish test split, same generation settings for both models. The metric is the one EuroEval ranks each dataset on (MCC, F1, or BERTScore).
Table with columns: Dataset, Metric, Base, This model, Change| Dataset | Metric | Base | This model | Change |
|---|
| AngryTweets (sentiment) | MCC | 38.88 | 41.70 | +2.82 |
| ScaLA-da (acceptability) | MCC | 31.28 | 31.13 | -0.15 |
| DANSK (NER) | F1 | 42.87 | 44.93 | +2.06 |
| MultiWikiQA-da (reading comp.) | F1 | 74.52 | 76.17 | +1.66 |
| Nordjylland News (summ.) | BERTScore |
Five datasets improve, two are unchanged within noise, and HellaSwag-da drops by 3.4 points. The largest gains are on sentiment, reading comprehension, and the citizenship-knowledge task.
Intended use and limitations
- Meant for Danish text tasks of the kind EuroEval covers. English ability is largely retained but was not the focus.
- HellaSwag-da (common-sense reasoning) is worse than the base model. If that task matters for your use, consider the base model or a lower merge strength.
- The idiom and civics tasks improved only a little. Supervised fine-tuning does not add new factual knowledge, so larger gains on knowledge-heavy tasks would require continued pretraining on Danish text.
- The base model's vision capabilities were neither trained nor evaluated.
- Results are single-run EuroEval scores. Per-dataset differences of a point or two are within the range of normal variation.
Training data and licensing
- Base model: Qwen3.5-2B (Apache-2.0).
- Danish source text: the Dynaword corpus (Danish Wikipedia, retsinformation, and other permissively licensed sources). The EuroEval benchmark datasets were excluded from training to avoid contamination.
- Synthetic instruction data: generated through the DeepSeek API, whose platform terms permit using outputs to train and release models.
- English data: FineWeb-Edu (ODC-By).
Released under Apache-2.0, following the base model. This is an independent fine-tune and is not affiliated with or endorsed by Alibaba, the Qwen team, or DeepSeek. "Qwen" is used only to identify the base model.