Intended use
- For: drafting SVA properties from specs for formal or simulation-based verification, inside a loop
that samples several candidates and checks them with a formal tool.
- Not for: single-shot generation that ships without verification. Single-shot pass@1 is about 45%
on human-written specs and 63% on machine-generated ones.
Quick start
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "mmoselhy077/qwen3-8b-sva-rest-r2-merged"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto") msgs = [{"role": "user", "content": "Write a SystemVerilog assertion: req must be granted within 3 cycles."}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=8192, do_sample=True, temperature=0.2)print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
Needs about 17 GB of GPU memory in bf16. Reasoning traces can be long, so leave max_new_tokens high.
Evaluation
FVEval-Verified, 356 problems (73 human-written, 283 machine-generated). A sample counts as correct only
if it is formally equivalent to the reference assertion, checked by a bidirectional miter in
Questa qverify. A vacuous assertion fails this check.
Table with columns: Metric, All, Human, Machine| Metric | All | Human | Machine |
|---|
| Equivalence pass@5 | 84.8% | 63.0% | 90.5% |
| Equivalence pass@1 (approx.) | — | ~45% | ~63% |
Caveats
- pass@5 is an oracle ceiling: it assumes you can pick the right sample. Our reference-free selector
does no better than random, so in practice you need a formal check to choose.
- The human split is small (n=73, about ±11 pp), so differences between family members are near noise.
- Do not use "proven" rates to compare these models. The harness counted vacuous proofs; the genuine
non-vacuous proof rate is about 8–11% for every model in the family.
Training
- SFT base: 839 curated NL→SVA examples (
sva-abl839-corpus).
- ReST round 1: the SFT model sampled K=20 answers per synthetic DUT prompt. Samples that
qverify
proved equivalent were kept (317, sva-rest-r1-synthetic).
- ReST round 2: human-style prompts written against real RTL from OpenTitan, CVA6, NVDLA, Ibex,
PicoRV32 and SERV, filtered the same way (117, upsampled ×3,
sva-rest-r2-humanlike).
Final run: LoRA r=64, α=128, 2 epochs on the union, merged into the base weights.
Model family
Table with columns: Model, Equiv@5, Human, Machine, Notes| Model | Equiv@5 | Human | Machine | Notes |
|---|
| rest-r2 (this) | 84.8% | 63.0% | 90.5% | Recommended |
| abl839 LoRA | 81.2% | 63.0% | 85.9% | SFT baseline, 698 MB adapter |
Limitations
- Trained and evaluated only on single-property, English NL→SVA tasks. Multi-property specs, liveness
under fairness constraints, and bind or checker files are untested.
- Gains came almost entirely on machine-written specs. Human-written specs remain the hard case.
- Outputs can be syntactically valid but semantically wrong, or vacuous. Always check them formally.
License
Apache-2.0, inherited from Qwen3-8B.