⚠️ Deprecated — preliminary (Phase 2 · The Stack v1)

This adapter is the original March-2026 hackathon (Phase 2) model, trained on bigcode/the-stack (v1, non-dedup). It is superseded by the paper's Phase 3 adapter, which was re-trained from scratch on the cleaner bigcode/the-stack-v2-dedup corpus. For paper-grade use, load the Phase 3 adapter from the umbrella repo:

python
Copy code
PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora", subfolder="tiny-aya-base/condition-1-en-5k-seed42")

This repo is kept for reproducibility of the preliminary results only — do not cite it for the paper. It was renamed from legesher/language-decoded-lora-condition-1-en-5k; the old URL continues to resolve via a Hugging Face redirect.

Raw English Python from bigcode/the-stack (v1, non-dedup), 5k subset. Tests whether code fine-tuning improves multilingual reasoning (replicates Aryabumi et al., 2024).

Part of the Language Decoded project (Cohere's Tiny Aya Expedition).

For the full adapter inventory across both phases, see the Language Decoded LoRA hub and its MANIFEST.md.

Training Data

legesher/language-decoded-data / phase-2-the-stack-v1-condition-1-en-5k — the Phase 2 / The Stack v1 config.

Usage

python
Copy code
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-base")
tokenizer = AutoTokenizer.from_pretrained("CohereLabs/tiny-aya-base")
# Preliminary Phase 2 adapter (kept for reproducibility):
model = PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora-phase-2-the-stack-v1-condition-1-en-5k")

Citation

bibtex
Copy code
@misc{language-decoded-2026,
  title={Language, Decoded: Exploring the Impact of Fine-Tuning a Multilingual Model on Native-Language Code},
  author={Madison Edgar and Saad Ahmed Bazaz and Tom Sherborne and Rashik Shahjahan and Khojasteh Mirza and Sarah Jawaid and Rafay Mustafa and Sohaib Ahmed Bazaz},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/legesher/language-decoded-lora}
}

License

Apache 2.0