vllm-project

smollm-135m-lora-fineweb-faroese

README

License: apache-2.0

[!NOTE] This is a copy of jekunz/smollm-135m-lora-fineweb-faroese at commit e55c7de, made on 2026-10-30. vLLM CI uses this checkpoint and the original is on a personal account, so we keep a copy here to reduce supply-chain risk.

This model is a SmolLM2-135M-Instruct model fine-tuned on the Faroese portion of Fineweb-2. It is intended for my research and has not been evaluated more broadly yet.

LoRA setup:

Rank: 256
Alpha: 512
Target modules: ["up_proj", "down_proj", "gate_proj", "o_proj"]

Training:

5 Epochs
Learning rate: 8e-4
LR scheduler: Cosine
Warmup ratio: 0.05
Batch size: 1
4 A100 (40GB) GPUs
Gradient accumulation steps: 64
Effective batch size: 256
Max. context length: 8192 tokens

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

vllm-project

Model Tree

Base

HuggingFaceTB/SmolLM2-135M-Instruct

Adapter

this model

Input Modalities

Text

Output Modalities