Dataset provenance
The synthetic generator was designed from Razorpay's public webhook documentation. The documented
behaviors used as source requirements are HMAC-SHA256 verification over the exact raw request body,
duplicate handling through the unique x-razorpay-event-id, at-least-once delivery, and tolerance
for out-of-order events. The generator created original Python modules and inserted controlled bugs;
it did not scrape Razorpay code or use private payloads.
Minor-unit typing, fulfil-once, delayed event commit, and unknown-event checks are additional
benchmark invariants. They should not be read as a description of Razorpay's internal code.
Source repository: https://github.com/nabeel-wq/payments-qwen-4b-finetued
Results
Candidates were compiled and executed against nine hidden behavioral checks. Base and tuned models
used the same prompts, renderer, temperature zero, one sample per task, and a 2,200-token cap.
Table with columns: Split, Qwen3.5-4B, PayFix LoRA| Split | Qwen3.5-4B | PayFix LoRA |
|---|
| One-defect test, 120 programs | 48.3% | 100% |
| Two-defect challenge, 24 programs | 33.3% | 95.8% |
The test and challenge sets use implementation-family IDs excluded from training. They are still
generated by the same program generator, so these numbers measure controlled domain adaptation and
should not be read as production correctness. The one tuned challenge failure accepted an empty
event ID while correctly repairing the other defect.
Full protocol, per-defect results, confidence intervals, and executable evaluator source are in the
PayFix GitHub repository.
Training
- Base:
Qwen/Qwen3.5-4B
- Platform: Tinker
- Method: LoRA SFT
- Rank: 16
- Epochs: 1
- Batch size: 8
- Optimizer learning rate:
1e-4, linear decay
- Steps: 75
- Training tokens reported by Tinker: 684,382
- Renderer:
qwen3_5_disable_thinking
- Loss: final assistant message only
No private code, credentials, customer payloads, or transaction records were used.
Loading the adapter
Use recent versions of Transformers and PEFT that support Qwen 3.5:
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_id = "Qwen/Qwen3.5-4B"
adapter_id = "rider-provider-777/payments-4b"
tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)
For vLLM, start the base model with the adapter attached:
vllm serve Qwen/Qwen3.5-4B \
--enable-lora \
--lora-modules payfix=rider-provider-777/payments-4b
Use the code-only system prompt and full behavioral contract from the GitHub repository. The adapter
was not trained as a general chat assistant.
Limitations
- Training and evaluation programs come from one deterministic generator.
- The state layer is an in-memory benchmark abstraction, not a real database transaction.
- The evaluation does not cover concurrent workers, framework middleware, network retries, secret
rotation, or the full Razorpay event schema.
- Generated code must be reviewed and tested before use. Do not execute untrusted generations
outside a hardened sandbox.
- The adapter inherits the capabilities, limitations, and license obligations of its base model.
License
Released under Apache-2.0. The base model is also published under Apache-2.0. Razorpay is referenced
only as the source of public webhook requirements; this project is not affiliated with or endorsed
by Razorpay.