What it does
The input is the KernelBench Triton one-shot prompt around a PyTorch Model with get_inputs() and get_init_inputs(). The model answers with one fenced Python block. That block defines ModelNew, which has the same interface and computes forward with @triton.jit kernels.
Results on KernelBench (Triton, levels 1 and 2)
The setup:
- KernelBench at revision
423217d, Triton backend.
- Levels 1 (100 single operators) and 2 (100 operator sequences to fuse).
- One greedy answer per problem: temperature 0, thinking off, at most 8,192 new tokens, vLLM with a BF16 base.
- FP32 with
atol = rtol = 1e-4.
- Correctness is checked on 5 random inputs.
- Timing is the mean of 100 CUDA-event runs, against eager PyTorch on the same GPU.
- Hardware and software: NVIDIA H100 80GB HBM3, PyTorch 2.9.1, Triton 3.5.1, CUDA 13.0.
How to read the table:
- fast_p is the share of the 200 problems where the answer is correct and more than p times faster than eager PyTorch.
- fast₀ is the correctness rate.
- Ran means the code compiled and ran without error.
Table with columns: Base model, After SFT, This adapter (SFT + RL) | Base model | After SFT | This adapter (SFT + RL) |
|---|
| Ran (L1 / L2) | 83 / 76 | 81 / 92 | 83 / 94 |
| Correct (L1 / L2) | 31 / 12 | 43 / 9 | 47 / 12 |
| Correct and faster than eager (L1 / L2) | 8 / 6 | 13 / 3 | 15 / 4 |
| fast₀ (correctness) | 21.5% | 26.0% | 29.5% |
| fast₁ | 7.0% | 8.0% | 9.5% |
| fast₁.₅ | 5.0% | 5.0% | 6.0% |
| fast₂ | 3.5% | 4.5% | 5.0% |
fast₁ against torch.compile(mode="max-autotune") | 12.5% | 15.0% | 16.0% |
What the numbers do and do not show:
- Correctness improved most: level 1 went from 31 to 47 correct kernels. Most of that gain came from SFT (31 → 43).
- The speed gain is small: from 14 to 19 of 200 problems faster than eager. With one answer per problem, differences of 1–3 problems are within noise. About 18 correct kernels per model run within 0.9–1.1× of eager, so they can land on either side of the threshold.
- Level 2 did not improve: 12 correct kernels before and after. Fast level 2 answers need real fusion, and the model rarely produces it.
- Correct is not the same as fast: the median correct kernel runs at 0.98× eager.
- The checkpoint was chosen on this benchmark: steps 100, 150, 200 and 300 scored fast₁ = 9.5%, 9.5%, 8.5% and 8.0%. Picking the best one makes the reported number slightly optimistic. Results come from one seed.
- Thinking helps correctness, not speed: see With thinking below.
- Do not compare directly with the KernelBench paper: it used CUDA kernels on an L40S. Papers also differ in level subsets, number of attempts, baselines and tolerances. Section 8 of our article lists what to check before comparing.
With thinking (medium)
We also scored this adapter with Qwen3.8's native reasoning on (reasoning_effort="medium"). That is the reasoning level of the frontier runs we compare against in the Part 2 article.
Settings: the same 200 problems and grader; one seeded answer per problem at temperature 0.6; 8,192 tokens for reasoning and answer together. Only the text after </think> is graded.
Table with columns: Without thinking (greedy), With thinking, medium, Change, pp [95% paired bootstrap] | Without thinking (greedy) | With thinking, medium | Change, pp [95% paired bootstrap] |
|---|
| Correct | 29.5% | 41.0% | +11.5 [+6.0; +17.0] |
| Correct (L1 / L2) | 47 / 12 | 59 / 23 | |
| fast₁ vs eager | 9.5% | 11.0% | +1.5 [−2.5; +5.5] |
fast₁ vs torch.compile | 16.0% | 18.0% | +2.0 [−1.5; +5.5] |
| Answers that hit the 8,192-token budget | — | 56 of 200 | |
- Reasoning raises correctness clearly, including level 2.
- It barely changes speed: level 2 fast answers are still pure Triton.
- More than a quarter of the answers ran out of tokens before the reasoning ended, so a larger budget would likely score higher.
- This is one seeded run.
How to use
vLLM
The numbers above were measured with vLLM. The vllm/ folder holds the same weights with tensor names that vLLM's Qwen3.8 model expects (model.language_model.layers.*). The root adapter uses model.layers.* for Transformers and PEFT. The two copies are otherwise byte-identical.
hf download g-factor-technologies/Qwen3.8-27B-GFT-Triton-Kernels-LoRA --include "vllm/*" --local-dir gft-kernelsvllm serve Qwen/Qwen3.8-27B --enable-lora --max-lora-rank 8 --max-model-len 16384 \ --lora-modules gft-kernels=./gft-kernels/vllm
Build the prompt with KernelBench at revision 423217d:
from kernelbench.prompt_constructor_toml import get_prompt_for_backendfrom openai import OpenAI reference = open("KernelBench/level1/19_ReLU.py").read()prompt = get_prompt_for_backend(reference, "triton", option="one_shot", precision="fp32") client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")reply = client.chat.completions.create( model="gft-kernels", messages=[{"role": "system", "content": "You are a helpful assistant"}, {"role": "user", "content": prompt}], temperature=0, max_tokens=8192, extra_body={"chat_template_kwargs": {"enable_thinking": False}},)print(reply.choices[0].message.content) # the first ```python block defines ModelNew
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base_id = "Qwen/Qwen3.8-27B"adapter_id = "g-factor-technologies/Qwen3.8-27B-GFT-Triton-Kernels-LoRA" tokenizer = AutoTokenizer.from_pretrained(base_id)model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, adapter_id) messages = [{"role": "system", "content": "You are a helpful assistant"}, {"role": "user", "content": prompt}] # prompt built as aboveinputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=8192, do_sample=False)print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
The adapter was trained without thinking, and the main numbers use thinking off. To use thinking:
- pass
"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "medium"};
- set
temperature=0.6 and a token budget of at least 8,192 (more is better);
- take the code from the text after
</think>, not from drafts inside the reasoning.
Always run generated kernels through your own correctness and timing checks before you use them.
Training
Table | |
|---|
| Base | Qwen/Qwen3.8-27B at revision 1d4bf0f |
| Adapter | LoRA r = 8, alpha = 16, no dropout, on q, k, v, o, gate, up and down projections |
| Stage 1: SFT | 2 epochs on GFT-Triton-Kernels-SFT-H100-Verified. Of its 2,034 rows, 10% were held out. Learning rate 2e-4, max length 4,096, seed 42 (3,674 steps) |
| Stage 2: RL | Starts from the SFT adapter. GRPO-style groups of 8 answers per task, one task per step, CISPO loss, REPO-R entropy control. Learning rate 1e-5, KL β = 0.08, temperature 1.0, up to 6,144 completion tokens, thinking off, 4-bit NF4 base (QLoRA) |
| Gym tasks | PyTorch references in 14 families: elementwise, reduction, normalization, fusion, matmul, softmax, GEMM epilogue, conv1d, conv2d, pool2d, scan, module, fused hybrid, and verified KernelBook modules. Hard families have easier staged variants. Some tasks use the KernelBench one-shot prompt and accept exactly one answer. The rest allow several submissions with evaluator feedback |
| Reward | Measured on an H100. To score, a kernel must compile and launch Triton kernels. PyTorch compute inside ModelNew is blocked; hybrid tasks allow one cuBLAS or cuDNN call. The output must match the reference on hidden test cases at FP32 tolerance (1e-4, scaled with input magnitude). Only correct kernels with stable timings earn speed credit, measured against torch.compile(mode="max-autotune") |
| Checkpoint | Step 100 of a 300-step RL run, chosen as described under Results |
| Compute | One NVIDIA H100 for SFT and one for RL |
Decontamination
- SFT data: we dropped every reference whose token-set Jaccard similarity to any of the 270 KernelBench problems (levels 1–4) is above 0.6.
- KernelBook modules in the gym: under the same measure, no module is above 0.6; the highest is 0.59.
- Generated gym families: these cover the same kinds of operations as KernelBench level 1 (ReLU, matmul, softmax, convolution and so on). Their code and shapes are generated, not copied from KernelBench.
Limitations
- One greedy answer per problem. Small differences between models are noise.
- Level 2 is weak. The adapter rarely fuses a library operation with its elementwise tail well enough to beat eager PyTorch.
- Correct is not fast. Half of the correct kernels are slower than eager PyTorch.
- Hardware-specific timings. Speedups were measured on one H100 and change with GPU, shape and software versions.
- FP32 and the KernelBench format only. Level 3 (full architectures) was not evaluated.
- Verify every kernel. Generated kernels can fail on shapes or values outside the tested ones. Treat each answer as a draft to check, not as a drop-in replacement.
Citation
@misc{gfactor2026tritonkernels, title = {Qwen3.8-27B-GFT-Triton-Kernels-LoRA}, author = {Romanov, Aleksei and {g factor technologies}}, year = {2026}, url = {https://huggingface.co/g-factor-technologies/Qwen3.8-27B-GFT-Triton-Kernels-LoRA}}@article{ouyang2025kernelbench, title = {KernelBench: Can LLMs Write Efficient GPU Kernels?}, author = {Ouyang, Anne and Guo, Simon and Arora, Simran and Zhang, Alex L. and Hu, William and R{\'e}, Christopher and Mirhoseini, Azalia}, journal = {arXiv preprint arXiv:2502.10517}, year = {2025}}
Built on Qwen3.8-27B by the Qwen team (Apache 2.0). The adapter is released under Apache 2.0. Part of the SFT data comes from third-party sources with their own terms; see the dataset card. Released by g factor technologies.