abhash-chakraborty

ophyx-350m-v2

README

License: apache-2.0

How it was made

Pretraining. 12 billion tokens of open data such as Stack-Edu and FineWeb-Edu, at 2,048 tokens of context, with a tokenizer and a JAX training loop written for this model.
Longer context. 2 billion more tokens at 4,096.
Fine-tuning. 800M tokens at 4,096: instructions, solutions to problems with tests, and bug-and-fix pairs.
Averaging. 70% of that fine-tune plus 30% of v1, weight by weight. The 4,096 run had lost ground on short tasks, and the average won it back.
Reinforcement learning. GRPO for one pass over 1,399 problems with tests. The only reward is passing them. 8 samples per problem, learning rate 2e-6, no KL term, 174 steps, 5.4 hours on one T4.

Results

Pass@1 in percent, greedy. Every model ran through the same harness and the same offline sandbox.

Table with columns: Model, HumanEval+, MBPP+, BCB-Hard (instruct), LiveCodeBench, CRUX-O, Hands-on (42)
Model	HumanEval+	MBPP+	BCB-Hard (instruct)	LiveCodeBench	CRUX-O	Hands-on (42)
Ophyx 350M v2	43.9	47.4	4.6	2.0	21.9	9
Ophyx 350M v1	35.4	48.1	3.8	1.5	19.5	10
SmolLM2-360M-Instruct	15.9	27.5	0.0	0.9	20.4	4
Qwen2.5-Coder-0.5B-Instruct	54.9	42.1	2.3	0.6	25.1	5
Qwen3-0.6B	36.6	46.6	0.8	7.9	24.8	6

The hands-on tasks are 42 small jobs closer to real work: data handling, classes, async code.

Beyond Python functions it falls further behind. It answers every JavaScript, C++ and Bash prompt in Python, and library-heavy tasks are hard at this size:

Table with columns: Model, MultiPL-E JS, MultiPL-E C++, MultiPL-E Bash, BigCodeBench instruct (1,140), DS-1000
Model	MultiPL-E JS	MultiPL-E C++	MultiPL-E Bash	BigCodeBench instruct (1,140)	DS-1000
Ophyx 350M v2	0.0	0.0	0.0	21.0	3.4
Qwen2.5-Coder-0.5B-Instruct	52.2	45.3	25.3	11.3	6.6
Qwen3-0.6B	30.4	21.1	5.1	10.1	5.3
deepseek-coder-1.3b-instruct	60.2	50.3	16.5	22.3	27.2
Qwen2.5-Coder-7B-Instruct	82.6	80.7	51.9	42.4	42.0

All 56 models, the frontier comparison and the hand-read answers are in the GitHub repo.

Use

The model answers in a fixed template and ends its answer with <|end|>. The tokenizer config needs transformers 5 or newer:

python

from transformers import AutoModelForCausalLM, AutoTokenizer repo = "abhash-chakraborty/ophyx-350m-v2"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo).eval() prompt = "Write a function `is_palindrome(s)` that ignores case and non-letters."ids = tok(f"<|user|>\n{prompt}\n<|python|>\n", return_tensors="pt", add_special_tokens=False).input_idsend = tok.convert_tokens_to_ids("<|end|>")out = model.generate(ids, max_new_tokens=256, do_sample=False, eos_token_id=end, pad_token_id=end)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

On transformers 4.x, load the tokenizer with PreTrainedTokenizerFast(tokenizer_file=hf_hub_download(repo, "tokenizer.json")) instead.

GGUF files for llama.cpp are in ophyx-350m-v2-gguf.

Limits

Python only, and English prompts only. Ask for JavaScript, C++ or Bash and it writes Python.
It writes short functions and modules well. Multi-file work and long reasoning are beyond it: it solves 2% of LiveCodeBench and 9 of the 42 hands-on tasks.
Library-heavy tasks are hard for it: 3.4 on DS-1000, most of it on Matplotlib.
It predicts function inputs (CRUX-I) worse than v1: 20.6 against 24.9.
Treat its output like any untested code.

Architecture

Qwen3 layout: 28 layers, hidden size 1,024, 16 query heads and 4 key-value heads of size 64, MLP 2,816, tied embeddings, a 32,768-token vocabulary, RoPE base 100,000 and 4,096 tokens of context. Weights are float32.

Licence

Apache-2.0.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

abhash-chakraborty

Model Tree

Base

this model

Input Modalities

Text

Output Modalities