Training
Table | |
|---|
| Base model | Qwen/Qwen3.5-4B |
| Algorithm | GRPO (leave-one-out baseline, group reward normalization, token-level loss). |
Generation budget (max_new_tokens) | 2,048 |
| Data | ExcelForum spreadsheet-manipulation tasks, 10 epochs max |
| Batch | 32 prompts × 8 rollouts per step |
| Optimizer | Adam, cosine LR schedule, peak LR 5e-7, 10 warmup steps |
| Steps | 100 |
| Reward | task checker on the produced spreadsheet |
| Weights dtype | BF16 |
Prompts are the ExcelForum spreadsheet tasks task prompts, rendered with the base model's chat template.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "budget-internalization-iclr2027/qwen3.5-4b-2k-grpo-spreadsheet-happyraven-s100"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
With vLLM: vllm serve budget-internalization-iclr2027/qwen3.5-4b-2k-grpo-spreadsheet-happyraven-s100
License
Inherits the license of the base model (Qwen/Qwen3.5-4B).