taylonmcfly

Qwen3.5-9B-Existence-Code

README

License: apache-2.0

Status

This release is an adapter release with training metadata and logs available locally from the training package. It is not yet a benchmarked research release.

What is documented:

Training data mix and example counts
Approximate training token count from trainer logs
LoRA/QLoRA hyperparameters
Training steps, runtime, checkpoints, and final training loss
Reproducible training script structure

What is not yet proven by this release:

It has not yet been independently benchmarked against the base model.
It has not yet been evaluated on HumanEval, MBPP, SWE-bench, translation benchmarks, or general capability regression tests.
It should not be claimed to be globally better than Qwen/Qwen3.5-9B-Base until a separate evaluation report is added.

Model Details

Model name: Qwen Existence Code
Repository name: Qwen3.5-9B-Existence-Code
Base model: Qwen/Qwen3.5-9B-Base
Model type: PEFT LoRA adapter
Training method: QLoRA SFT
Adapter file: adapter_model.safetensors
Adapter size: about 58 MB
License: Apache-2.0
Primary languages: English and Russian

Files

text

README.mdadapter_model.safetensorsadapter_config.jsontokenizer.jsontokenizer_config.jsonchat_template.jinjatraining_args.bin

Intended Use

Typical use cases:

Code generation and code completion
Refactoring and debugging assistance
Technical documentation
RAG-based project assistants
Automation planning
Game localization and dialogue translation
Context-aware translation where tone, slang, mature language, or character voice must be preserved

For localization tasks, the adapter is intended to preserve the meaning and tone of source material when the user has the right to process that content. Fictional game dialogue may include profanity, slang, dark humor, or mature language, and removing it can damage the original intent.

Training Data

The training set was built as a mixed SFT JSONL file:

text

data/sft_mix.jsonl

Observed dataset statistics:

Table with columns: Source, Examples
Source	Examples
`microsoft/orca-agentinstruct-1M-v1`	500
`HuggingFaceH4/ultrafeedback_binarized`	250
`nvidia/Nemotron-SFT-OpenCode-v1`	250
`synthetic_complete_code`	80
Total	1080

Approximate text size:

text

examples: 1080characters: 3,985,263average characters/example: 3,690.1

The trainer log reported approximately:

text

num_tokens: 3.899e+06

at the final training step.

Dataset Filtering

The dataset preparation script filtered out examples containing placeholder or incomplete-answer patterns, including:

text

TODOinsert your code hereyour code hereleft as an exerciseyou can continueand so onetc....implement the restfill inplaceholder

Synthetic examples were added to emphasize complete runnable code and avoidance of placeholder-only answers.

Training Procedure

Training used QLoRA supervised fine-tuning.

Base model path during training:

text

/workspace/train/qwen35_diligent_lora/models/Qwen3.5-9B-Base

Public base model:

text

Qwen/Qwen3.5-9B-Base

LoRA Parameters

From adapter_config.json:

text

peft_type: LORAtask_type: CAUSAL_LMr: 16lora_alpha: 32lora_dropout: 0.05bias: noneuse_dora: falseuse_rslora: false

Target modules:

text

q_projk_projv_projo_projgate_projup_projdown_proj

Quantization

Training used 4-bit QLoRA:

text

load_in_4bit: truebnb_4bit_quant_type: nf4bnb_4bit_compute_dtype: bfloat16bnb_4bit_use_double_quant: true

SFT Hyperparameters

text

max_steps: 600save_steps: 100max_length / sequence length: 4096per_device_train_batch_size: 1gradient_accumulation_steps: 8effective batch size: 8learning_rate: 1.5e-4warmup_ratio: 0.03lr_scheduler_type: cosinelogging_steps: 5save_total_limit: 6bf16: truefp16: falsegradient_checkpointing: trueoptimizer: paged_adamw_8bitseed: 42packing: false

Runtime

Final trainer log:

text

train_runtime: 6914 secondstrain_samples_per_second: 0.694train_steps_per_second: 0.087epoch: 4.444

Loss Curve

The final training loss was:

text

train_loss: 0.4767

Selected logged points from the final stage:

Table with columns: Step, Epoch, Loss, Mean token accuracy, Learning rate
Step	Epoch	Loss	Mean token accuracy	Learning rate
540	4.000	0.3872	0.8982	4.029e-06
545	4.037	0.2483	0.9253	3.401e-06
550	4.074	0.3037	0.9060	2.824e-06
560

Interpretation: the loss indicates that the adapter fit the SFT mixture, but training loss alone does not prove downstream quality. External evaluation is still required.

Evaluation

No formal benchmark evaluation is included in this release yet.

Recommended evaluation before making quality claims:

Code

HumanEval
MBPP
LiveCodeBench
Repo-level patch tests
Internal unit-test based coding tasks

Translation and Localization

Human review on game dialogue samples
Terminology consistency tests
Format preservation tests
Variable/tag preservation tests
Side-by-side comparison with the base model

General Regression

MMLU-style knowledge checks
GSM/math samples
Summarization and instruction-following checks
Refusal/safety behavior checks if deployed publicly

Until these tests are added, the correct claim is:

text

This adapter was trained to bias Qwen3.5-9B-Base toward more complete technical and localization-style outputs, but benchmarked improvements over the base model have not yet been established.

Reproducibility

The original training package included:

text

data/sft_mix.jsonlscripts/prepare_sft_mix.pyscripts/train_qlora.pyscripts/start_train_qlora.shlogs/train_qwen35_diligent_lora_*.logoutputs/qwen35_diligent_lora_v1/checkpoint-100outputs/qwen35_diligent_lora_v1/checkpoint-200outputs/qwen35_diligent_lora_v1/checkpoint-300outputs/qwen35_diligent_lora_v1/checkpoint-400outputs/qwen35_diligent_lora_v1/checkpoint-500outputs/qwen35_diligent_lora_v1/checkpoint-600outputs/qwen35_diligent_lora_v1/final_adapter

Minimal reproduction command:

bash

source /venv/main/bin/activateCUDA_VISIBLE_DEVICES=0 python scripts/train_qlora.py \  --model /path/to/Qwen3.5-9B-Base \  --dataset data/sft_mix.jsonl \  --output outputs/qwen35_diligent_lora_v1 \  --max-steps 600 \  --save-steps 100 \  --seq-len 4096 \  --lr 1.5e-4

Loading With Transformers + PEFT

python

import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base_model = "Qwen/Qwen3.5-9B-Base"adapter_path = "taylonmcfly/Qwen3.5-9B-Existence-Code" tokenizer = AutoTokenizer.from_pretrained(adapter_path, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained(    base_model,    torch_dtype=torch.bfloat16,    device_map="auto",    trust_remote_code=True,)model = PeftModel.from_pretrained(model, adapter_path)model.eval() messages = [    {        "role": "system",        "content": "You are Qwen Existence Code, a precise technical assistant.",    },    {        "role": "user",        "content": "Write a complete Python script that scans a folder and prints file sizes.",    },] text = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,)inputs = tokenizer(text, return_tensors="pt").to(model.device) with torch.no_grad():    output = model.generate(        **inputs,        max_new_tokens=1024,        temperature=0.4,        top_p=0.9,        repetition_penalty=1.1,    ) print(tokenizer.decode(output[0], skip_special_tokens=True))

Recommended Prompting

text

You are Qwen Existence Code, a precise technical assistant.Follow the user's task completely.When writing code, provide complete runnable files or patches unless the user explicitly asks for a sketch.Do not replace required logic with placeholders such as TODO, "insert code here", or "continue yourself".Use the provided RAG context as the source of truth when it is relevant.For translation/localization, preserve meaning, tone, character voice, formatting, variables, markup, and speaker intent.

Recommended Generation Settings

Technical work:

text

temperature: 0.2-0.5top_p: 0.85-0.95repetition_penalty: 1.05-1.15max_new_tokens: high enough for complete output

Translation/localization:

text

temperature: 0.3-0.7top_p: 0.9repetition_penalty: 1.05

Limitations

This is a LoRA adapter, not a full standalone model.
It depends on Qwen/Qwen3.5-9B-Base.
It can hallucinate or make mistakes.
Training loss does not prove real-world superiority.
Formal code, translation, and regression benchmarks are not included yet.
Human review is recommended for production code and sensitive translations.
It may require RAG context for project-specific facts.

Responsible Use

This model is a tool. Users are responsible for their inputs, outputs, and deployment choices.

Do not use this model for illegal activity, unauthorized access, fraud, targeted harassment, or other harmful purposes. For legal, medical, financial, security-critical, or production-sensitive work, outputs should be reviewed by a qualified human.

Citation

text

Qwen Existence Code, PEFT LoRA adapter for Qwen/Qwen3.5-9B-Base.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

taylonmcfly

Model Tree

Base

Qwen/Qwen3.5-9B-Base

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Status

This release is an adapter release with training metadata and logs available locally from the training package. It is not yet a benchmarked research release.

What is documented:

Training data mix and example counts
Approximate training token count from trainer logs
LoRA/QLoRA hyperparameters
Training steps, runtime, checkpoints, and final training loss
Reproducible training script structure

What is not yet proven by this release:

It has not yet been independently benchmarked against the base model.
It has not yet been evaluated on HumanEval, MBPP, SWE-bench, translation benchmarks, or general capability regression tests.
It should not be claimed to be globally better than Qwen/Qwen3.5-9B-Base until a separate evaluation report is added.

Model Details

Model name: Qwen Existence Code
Repository name: Qwen3.5-9B-Existence-Code
Base model: Qwen/Qwen3.5-9B-Base
Model type: PEFT LoRA adapter
Training method: QLoRA SFT
Adapter file: adapter_model.safetensors
Adapter size: about 58 MB
License: Apache-2.0
Primary languages: English and Russian

Files

text

README.mdadapter_model.safetensorsadapter_config.jsontokenizer.jsontokenizer_config.jsonchat_template.jinjatraining_args.bin

Intended Use

Typical use cases:

Code generation and code completion
Refactoring and debugging assistance
Technical documentation
RAG-based project assistants
Automation planning
Game localization and dialogue translation
Context-aware translation where tone, slang, mature language, or character voice must be preserved

Training Data

The training set was built as a mixed SFT JSONL file:

text

data/sft_mix.jsonl

Observed dataset statistics:

Table with columns: Source, Examples
Source	Examples
`microsoft/orca-agentinstruct-1M-v1`	500
`HuggingFaceH4/ultrafeedback_binarized`	250
`nvidia/Nemotron-SFT-OpenCode-v1`	250
`synthetic_complete_code`	80
Total	1080

Approximate text size:

text

examples: 1080characters: 3,985,263average characters/example: 3,690.1

The trainer log reported approximately:

text

num_tokens: 3.899e+06

at the final training step.

Dataset Filtering

The dataset preparation script filtered out examples containing placeholder or incomplete-answer patterns, including:

text

TODOinsert your code hereyour code hereleft as an exerciseyou can continueand so onetc....implement the restfill inplaceholder

Synthetic examples were added to emphasize complete runnable code and avoidance of placeholder-only answers.

Training Procedure

Training used QLoRA supervised fine-tuning.

Base model path during training:

text

/workspace/train/qwen35_diligent_lora/models/Qwen3.5-9B-Base

Public base model:

text

Qwen/Qwen3.5-9B-Base

LoRA Parameters

From adapter_config.json:

text

peft_type: LORAtask_type: CAUSAL_LMr: 16lora_alpha: 32lora_dropout: 0.05bias: noneuse_dora: falseuse_rslora: false

Target modules:

text

q_projk_projv_projo_projgate_projup_projdown_proj

Quantization

Training used 4-bit QLoRA:

text

load_in_4bit: truebnb_4bit_quant_type: nf4bnb_4bit_compute_dtype: bfloat16bnb_4bit_use_double_quant: true

SFT Hyperparameters

text

max_steps: 600save_steps: 100max_length / sequence length: 4096per_device_train_batch_size: 1gradient_accumulation_steps: 8effective batch size: 8learning_rate: 1.5e-4warmup_ratio: 0.03lr_scheduler_type: cosinelogging_steps: 5save_total_limit: 6bf16: truefp16: falsegradient_checkpointing: trueoptimizer: paged_adamw_8bitseed: 42packing: false

Runtime

Final trainer log:

text

train_runtime: 6914 secondstrain_samples_per_second: 0.694train_steps_per_second: 0.087epoch: 4.444

Loss Curve

The final training loss was:

text

train_loss: 0.4767

Selected logged points from the final stage:

Table with columns: Step, Epoch, Loss, Mean token accuracy, Learning rate
Step	Epoch	Loss	Mean token accuracy	Learning rate
540	4.000	0.3872	0.8982	4.029e-06
545	4.037	0.2483	0.9253	3.401e-06
550	4.074	0.3037	0.9060	2.824e-06
560

Interpretation: the loss indicates that the adapter fit the SFT mixture, but training loss alone does not prove downstream quality. External evaluation is still required.

Evaluation

No formal benchmark evaluation is included in this release yet.

Recommended evaluation before making quality claims:

Code

HumanEval
MBPP
LiveCodeBench
Repo-level patch tests
Internal unit-test based coding tasks

Translation and Localization

Human review on game dialogue samples
Terminology consistency tests
Format preservation tests
Variable/tag preservation tests
Side-by-side comparison with the base model

General Regression

MMLU-style knowledge checks
GSM/math samples
Summarization and instruction-following checks
Refusal/safety behavior checks if deployed publicly

Until these tests are added, the correct claim is:

text

This adapter was trained to bias Qwen3.5-9B-Base toward more complete technical and localization-style outputs, but benchmarked improvements over the base model have not yet been established.

Reproducibility

The original training package included:

text

data/sft_mix.jsonlscripts/prepare_sft_mix.pyscripts/train_qlora.pyscripts/start_train_qlora.shlogs/train_qwen35_diligent_lora_*.logoutputs/qwen35_diligent_lora_v1/checkpoint-100outputs/qwen35_diligent_lora_v1/checkpoint-200outputs/qwen35_diligent_lora_v1/checkpoint-300outputs/qwen35_diligent_lora_v1/checkpoint-400outputs/qwen35_diligent_lora_v1/checkpoint-500outputs/qwen35_diligent_lora_v1/checkpoint-600outputs/qwen35_diligent_lora_v1/final_adapter

Minimal reproduction command:

bash

source /venv/main/bin/activateCUDA_VISIBLE_DEVICES=0 python scripts/train_qlora.py \  --model /path/to/Qwen3.5-9B-Base \  --dataset data/sft_mix.jsonl \  --output outputs/qwen35_diligent_lora_v1 \  --max-steps 600 \  --save-steps 100 \  --seq-len 4096 \  --lr 1.5e-4

Loading With Transformers + PEFT

python

import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base_model = "Qwen/Qwen3.5-9B-Base"adapter_path = "taylonmcfly/Qwen3.5-9B-Existence-Code" tokenizer = AutoTokenizer.from_pretrained(adapter_path, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained(    base_model,    torch_dtype=torch.bfloat16,    device_map="auto",    trust_remote_code=True,)model = PeftModel.from_pretrained(model, adapter_path)model.eval() messages = [    {        "role": "system",        "content": "You are Qwen Existence Code, a precise technical assistant.",    },    {        "role": "user",        "content": "Write a complete Python script that scans a folder and prints file sizes.",    },] text = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,)inputs = tokenizer(text, return_tensors="pt").to(model.device) with torch.no_grad():    output = model.generate(        **inputs,        max_new_tokens=1024,        temperature=0.4,        top_p=0.9,        repetition_penalty=1.1,    ) print(tokenizer.decode(output[0], skip_special_tokens=True))

Recommended Prompting

text

You are Qwen Existence Code, a precise technical assistant.Follow the user's task completely.When writing code, provide complete runnable files or patches unless the user explicitly asks for a sketch.Do not replace required logic with placeholders such as TODO, "insert code here", or "continue yourself".Use the provided RAG context as the source of truth when it is relevant.For translation/localization, preserve meaning, tone, character voice, formatting, variables, markup, and speaker intent.

Recommended Generation Settings

Technical work:

text

temperature: 0.2-0.5top_p: 0.85-0.95repetition_penalty: 1.05-1.15max_new_tokens: high enough for complete output

Translation/localization:

text

temperature: 0.3-0.7top_p: 0.9repetition_penalty: 1.05

Limitations

This is a LoRA adapter, not a full standalone model.
It depends on Qwen/Qwen3.5-9B-Base.
It can hallucinate or make mistakes.
Training loss does not prove real-world superiority.
Formal code, translation, and regression benchmarks are not included yet.
Human review is recommended for production code and sensitive translations.
It may require RAG context for project-specific facts.

Responsible Use

This model is a tool. Users are responsible for their inputs, outputs, and deployment choices.

Citation

text

Qwen Existence Code, PEFT LoRA adapter for Qwen/Qwen3.5-9B-Base.