AlexThunder0

qwen-cobol-27b

README

License: apache-2.0

Usage

The adapter keys are canonical PEFT format, so it loads with standard peft — no patching needed:

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.6-27B", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "AlexThunder0/qwen-cobol-27b")
tok = AutoTokenizer.from_pretrained("AlexThunder0/qwen-cobol-27b")

prompt = '''Complete the following COBOL program by implementing the PROCEDURE DIVISION.
The code MUST compile with the open-source GnuCOBOL compiler. Declare every variable
in WORKING-STORAGE, do not use IBM-only intrinsics, end with exactly one END PROGRAM.
Provide the complete, compilable COBOL program inside a single ```cobol code block.

```cobol
<the COBOLEval-style skeleton with a LINKAGE SECTION and a RESULT field>
```'''

msg = tok.apply_chat_template([{"role": "user", "content": prompt}],
                              tokenize=False, add_generation_prompt=True, enable_thinking=False)
out = model.generate(**tok(msg, return_tensors="pt").to(model.device),
                     max_new_tokens=2048, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))

The model expects the COBOLEval sub-program format: a skeleton with a LINKAGE SECTION and a RESULT field, which it completes with the WORKING-STORAGE SECTION and PROCEDURE DIVISION, ending in GOBACK / END PROGRAM. Greedy decoding is what the reported numbers use.

How it was made

Base: Qwen3.6-27B (open weights)
Adapter: DoRA, rank 128, α=256, rsLoRA, 7 target projections; 2 epochs, bf16, single H100
Data: a runtime-verified pipeline — every training example is compiled and executed with GnuCOBOL and checked against expected outputs before inclusion; decontaminated against COBOLEval (program-id / entry-point / n-gram overlap). The dataset itself is not released.

Full write-up: https://github.com/AlexThunder01/qwenCobol

Limitations

Below SOTA on functional correctness (Pass@1 36.3% vs 49.3%). The model reliably writes valid, compilable COBOL (88.4%); of the 146 COBOLEval problems, 76 compile but give the wrong answer — mostly genuinely-wrong algorithms (52), inverted boolean conditions (14), and off-by-one errors (7).
Tuned for the COBOLEval sub-program format and GnuCOBOL dialect. It may not follow other COBOL conventions (mainframe/Enterprise dialects, file I/O heavy programs).
Greedy decoding recommended for this task; in earlier testing on this pipeline, the base model's "thinking" mode gave only a marginal Pass@1 gain over greedy direct decoding, at much higher inference cost.

Citation / references

Benchmark: COBOLEval (BloopAI)
SOTA reference: COBOL-Coder (arXiv:2604.03986)
Base model: Qwen/Qwen3.6-27B

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

AlexThunder0

Model Tree

Base

Qwen/Qwen3.6-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality