Files
Table with columns: File, Description| File | Description |
|---|
model-*.safetensors + config/tokenizer | Final model (base + SFT + DPO, merged) in bf16 (~30 GB), for transformers, vLLM, etc. |
rosa-Q4_K_M.gguf | Same model quantized to Q4_K_M (~8.4 GB), for Ollama / llama.cpp |
Modelfile | Ollama Modelfile with chat template, system prompt and sampling defaults |
system_prompt.md | The Rosa persona system prompt used in training and at inference |
system_prompt_verbose.md | Alternative prompt for longer, multi-paragraph answers (use with ~1024 max new tokens) |
Quick start
Ollama
hf download marcoeg/rosa-qwen3-14b rosa-Q4_K_M.gguf Modelfile --local-dir rosacd rosa && ollama create rosa -f Modelfile && ollama run rosa
llama.cpp
llama-cli -m rosa-Q4_K_M.gguf -cnv \ --temp 0.7 --top-p 0.9 --top-k 50 -c 8192 \ -sys "$(cat system_prompt.md)"
from huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "marcoeg/rosa-qwen3-14b"system = open(hf_hub_download(repo, "system_prompt.md")).read()tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto") messages = [{"role": "system", "content": system}, {"role": "user", "content": "What is a general strike?"}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7, top_p=0.9, top_k=50)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
bf16 needs about 30 GB of GPU memory.
Recommended settings: temperature 0.7, top_p 0.9, top_k 50, context 8192.
Rosa was trained with the system prompt in system_prompt.md, so use it for the best results. For longer, more developed answers, use system_prompt_verbose.md instead. It works, but it pulls against the short style Rosa was trained on.
Thinking mode is turned off (/no_think).
Hardware for the GGUF: Q4_K_M runs in about 10 GB of VRAM/unified memory. It fits on a 12–16 GB GPU or an Apple Silicon Mac.
Try it in the browser: marcoeg/rosa-chat.
Training
Table with columns: Phase, Method, Data, Notes| Phase | Method | Data | Notes |
|---|
| 1. SFT | QLoRA (NF4), TRL SFTTrainer | ShareGPT conversations generated from a ~1.5M-word corpus of 51 historical and theoretical texts on anarcho-syndicalism and the labor movement | LoRA r=16, α=32 on all attention + MLP projections; source-disjoint train/eval split |
| 2. DPO | QLoRA (NF4), TRL DPOTrainer | Self-generated preference pairs: chosen = Rosa-voiced answer, rejected = base Qwen3-14B generic answer | β=0.1, sigmoid loss; trained on top of the merged SFT model |
| Export | merge → convert_hf_to_gguf.py → llama-quantize Q4_K_M | | |
The DPO adapter was merged into the merged SFT model. The result is published in bf16 safetensors and as a Q4_K_M GGUF converted with llama.cpp.
Intended use
- Exploring the history and ideas of the labor movement, anarcho-syndicalism, cooperatives and mutual aid
- Writing persona-driven commentary, organizing material and educational dialogue
- Research on persona and value-alignment fine-tuning
Limitations and risks
- Ideological by design. Rosa's analysis is partisan on purpose. Don't use it where neutrality is required.
- Hallucination. Like any 14B model, and more so after 4-bit quantization, Rosa can get historical
facts, dates and quotes wrong. Check claims before you rely on them.
- Not a real person. Rosa doesn't impersonate any historical figure. If you deploy it as an agent,
tell users they are talking to an AI.
- English only was evaluated.
License
Model weights are released under Apache-2.0, the same license as the Qwen3-14B base model. The
third-party texts in the training corpus keep their original licenses. The corpus itself is not
redistributed here.