Gemma-4-26B-A4B-sigma-rules API & Inference Endpoint

Model details

Table

Developed by	SASVA AI Model Cognition Labs (MCL) Team
Base model	`google/gemma-4-26B-A4B-it`
Base revision	`4d7ae4984b7db7de8f8457170b3f1a419ee76d52`
Base parameters	25,805,936,206 total (Hub safetensors metadata); mixture-of-experts, ~4B active per token
Architecture family	`gemma4` (`Gemma4ForConditionalGeneration`; text tower with 30 layers, 128 experts, hidden size 2816, vocabulary 262,144)
Adaptation	LoRA (`r=32`, `alpha=64`, `dropout=0.05`, rsLoRA off, DoRA off), merged into the root weights; adapter kept under `adapter/`
Trainable modules	`q_proj`, `k_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` on all 30 layers; `v_proj` on the 25 sliding-window layers (see below)
Excluded modules	`.vision_tower.` (the vision tower is untouched; training and evaluation were text-only)
Training method	`qlora` (`--load-in-4bit`, run 8 / trial 7)
Refinement	none
Precision	training: 4-bit NF4 base with double quantisation, bf16 compute, adapter in float32; root weights: bfloat16 merge
Language	English
License	Apache-2.0 (inherited from the base model); training rules are DRL 1.1

Trainable parameters: 37,171,200 across 205 modules, 0.1438% of the 25,843,107,406 parameters with the adapter attached. adapter/adapter_model.safetensors is 148,745,744 bytes (410 tensors, lora_A + lora_B per module, all float32). Every tensor sits under base_model.model.model.language_model.

One Gemma 4 structural fact shapes the module list. Confirmed against the base model's config.json: the 30 text layers alternate 5 sliding-window attention layers (window 1024, 16 heads over 8 KV heads, head size 256) with one full-attention layer, so layers 5, 11, 17, 23 and 29 are global attention. The global layers use 2 key-value heads of size 512 and have no v_proj weight at all (verified against the merged model's safetensors index: layer 5 carries q_proj, k_proj, o_proj, q_norm, k_norm only). PEFT therefore attached v_proj adapters to 25 layers and the other six projections to 30. The tensor counts match exactly: 60 per projection for 30 layers × A/B, 50 for v_proj.

Intended use

Direct use. Draft a Sigma rule from a written detection requirement for a detection engineer to review, validate with pySigma and adapt. The rule body is the product; the model also emits level and ATT&CK tags but these were not evaluated.

The model was trained on a specific prompt shape and that shape is part of the contract:

System prompt (verbatim): "You are a Sigma rule generator. Given a plain-language detection requirement, output a valid Sigma detection rule in YAML format. Start with title: and include all standard Sigma fields (title, status, description, logsource, detection, level, and any relevant fields/tags). Output only the raw YAML with no code fences, no explanations, and no additional text."
User turn: the instruction "You are a detection engineer. Write a valid Sigma rule (YAML) that satisfies the requirement. Output only the YAML.", a blank line, then the requirement block inside a bare ``` fence. This is the exact string the evaluator rendered (instruction + "\n\n```\n" + requirement_block + "\n```").
The requirement block is one to four lines, in this order and with this wording: an optional Log source: <product> / <category>. line, the mandatory Requirement: <description> line, an optional ATT&CK: T1059.003, T1218.011. line, and an optional Known false positives: <a>; <b>. line. In training, each optional line was present with probability 0.75 / 0.6 / 0.5 respectively, so the model handles both terse and detailed requests.
Applied through the tokenizer's chat template (chat_template.jinja, shipped in this repo) with . Do not concatenate strings by hand.

Out-of-scope use. Deploying an unreviewed rule to a SIEM. Generating rules for log sources absent from SigmaHQ. Any use as a detector of the threats the rules describe.

How to get started

The prompt pieces are the same in every path. Verbatim from the training invocation; do not paraphrase.

python

SYSTEM = (    "You are a Sigma rule generator. Given a plain-language detection requirement, "    "output a valid Sigma detection rule in YAML format. Start with `title:` and "    "include all standard Sigma fields (title, status, description, logsource, "    "detection, level, and any relevant fields/tags). Output only the raw YAML with "    "no code fences, no explanations, and no additional text.")INSTRUCTION = (    "You are a detection engineer. Write a valid Sigma rule (YAML) that satisfies "    "the requirement. Output only the YAML.")requirement = (    "Log source: windows / process_creation.\n"    "Requirement: Detects the execution of the hacktool Rubeus via PE information "    "or command line parameters\n"    "ATT&CK: T1003, T1558.003.")messages = [    {"role": "system", "content": SYSTEM},    {"role": "user", "content": f"{INSTRUCTION}\n\n```\n{requirement}\n```"},]

1. Fused weights with transformers (about 52 GB of accelerator memory in bfloat16; this is the exact directory the evaluation was served from):

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer REPO = "SASVAAI/Gemma-4-26B-A4B-sigma-rules"# AutoModelForCausalLM resolves to Gemma4ForConditionalGeneration on# transformers 5.7 and loads the text model.tokenizer = AutoTokenizer.from_pretrained(REPO)model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")model.eval() inputs = tokenizer.apply_chat_template(    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True).strip())

2. Fused weights with vLLM (how the numbers below were produced; the tensor-parallel degree must divide the 16 attention heads):

bash

pip install "vllm>=0.19.1"vllm serve SASVAAI/Gemma-4-26B-A4B-sigma-rules --tensor-parallel-size 4 --max-model-len 4096

python

from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")r = client.chat.completions.create(    model="SASVAAI/Gemma-4-26B-A4B-sigma-rules",    messages=messages, temperature=0, max_tokens=1024,)print(r.choices[0].message.content)

3. Adapter on the base model (base in 4-bit fits one 24 GB-class GPU, about 16 GB; verified on CPU in bf16 with these exact calls):

python

import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer BASE = "google/gemma-4-26B-A4B-it"REPO = "SASVAAI/Gemma-4-26B-A4B-sigma-rules"tokenizer = AutoTokenizer.from_pretrained(REPO, subfolder="adapter")model = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, REPO, subfolder="adapter")model.eval()# then generate exactly as in path 1

For the 4-bit base pass

text

quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)

, which reproduces the training-time numerics. vLLM cannot attach a LoRA to this architecture (its Gemma4ForConditionalGeneration does not list LoRA support), which is why the fused weights are at the root.

Actual output of path 3 for the prompt above (CPU, bf16, greedy), first lines:

text

title: HackTool - Rubeus Executiondescription: Detects the execution of the hacktool Rubeus via PE information or command line parameterslogsource:  category: process_creation  product: windowsdetection:  selection_img:    OriginalFileName: Rubeus.exe  selection_cli:    CommandLine|contains:    - ' /ticket'    - ' /ptt'    - ' /asktgt'    - ' /askns'    - ' /ptt'    - ' /ptt'      <- the repetition loop described under Evaluation;    ...               this prompt is one of the runaway cases.

Decoding matters. The metric was scored greedily with max_new_tokens=2048 through the chat template. No sampling setting was validated.

Training details

Data. 3,371 training rules and 375 validation rules built from the SigmaHQ/sigma repository at commit b1512572c56dbcc4e083ac0cd7e19f266ba52644 (Detection Rule License 1.1) by a deterministic script (autocatalyst.datagen.sigma_rules, seed 0). No model generated any training content.

Row construction, as recorded in the builder's manifest:

Source directories rules/, rules-threat-hunting/ and rules-emerging-threats/; rules-compliance/ skipped. 3,757 rules read, 11 dropped for exceeding 8,000 characters, 3,746 kept.
Input carries the rule's description as the requirement and, per rule and deterministically from the seed, sometimes the log source (75%), the ATT&CK technique ids parsed from tags (60%) and the falsepositives list (50%). It never contains the rule's title or detection block, which are the answer. In the 375 validation rows: 282 carry a log source, 206 an ATT&CK line, 79 a false-positives line.
Output is the rule re-serialised with only title, description, logsource, , , , , in that order. , , , , , , and are dropped as unlearnable noise.

Table

Train samples	3,371 rules
Validation samples	375 rules (10% of groups)
Group overlap	0 groups
Prompt format	chat template + system prompt + instruction / fenced-requirement user turn (see Intended use)
Loss masking	answer tokens only; prompt tokens set to `-100`
Truncation	sequences cut to `max_seq_len` 2048; a 7.5 kB rule is about 2,000 tokens, so the longest targets lose their tail during training

An LLM (Claude Opus 4.6, claude-opus-4-6, via an internal inference gateway) proposed the hyperparameters the search tried and wrote the system prompt from the project's problem statement. It generated no training content and computed no metric.

Method

Table

SFT method	`qlora`
Base quantisation during training	4-bit NF4, double quantisation, bf16 compute (`bitsandbytes`)
Refinement stage	none
Auto class	`AutoModelForCausalLM` (resolves to `Gemma4ForConditionalGeneration`)
Hardware	6x NVIDIA H100 80GB HBM3, `torchrun --nproc_per_node=6`

The project allowed one method (qlora). No refinement stage ran; the published adapter is the SFT adapter and the root weights are its merge.

Final hyperparameters

Table with columns: Hyperparameter, Value, Source
Hyperparameter	Value	Source
`learning_rate`	0.0002	`[TRAIN]` cmdline
`lr_scheduler_type`	cosine	`[TRAIN]` cmdline
`num_train_epochs`	4	`[TRAIN]` cmdline, `adapter/trainer_state.json`

Effective batch size: 24 (1 x 4 x 6). Optimizer steps: 564 (141 per epoch).

neftune_noise_alpha (0.0), use_dora, use_rslora and lora_init (default) were left at their no-op defaults. KD parameters are omitted deliberately: this is a qlora run, not a distillation run.

Provenance note: every value above was recovered from the platform database (runs, experiments, events tables for run 8) and cross-checked against the literal [TRAIN] command line recorded in the run log and against the shipped adapter/adapter_config.json.

How these values were chosen

These hyperparameters were selected by an automated search (autocatalyst.cli.run_autoresearch): an agent proposes one change at a time, runs train then eval, and keeps or discards on rouge_l (higher is better).

Run 8 ran 12 trials in 22 h 21 m (2026-09-24 20:36 to 2026-09-25 18:56 UTC); 11 scored and 1 errored. This checkpoint is trial 7, the run's best. Every trial trained on the same 3,371 rows and was scored on the same 375 validation rows, so the whole table is one comparison.

Table with columns: #, r, Dropout, LR, Epochs, Seq len, Weight decay, Train, Eval, rouge_l, Kept
#	r	Dropout	LR	Epochs	Seq len	Weight decay	Train	Eval	rouge_l	Kept
1	16	0.05	2e-4	2	4096	0.01	3 h 00 m	–	error

All trials: QLoRA, lora_alpha = 2 x r, cosine schedule, batch 1 per device, gradient accumulation 4, warmup 0.05, LoRA+ off, rsLoRA and DoRA off.

What the search actually established.

Epochs mattered up to 4. With everything else fixed at r=16, 2 → 3 → 4 epochs moved ROUGE-L 0.5238 → 0.5427 → 0.5767 (#2, #3, #4); 5 epochs fell back to 0.5427 (#5).
Rank 16 → 32 helped; 64 did not help further. #4 → #7 moved 0.5767 → 0.5925; #9 at r=64 scored 0.5888, inside the re-run spread below.
Dropout, learning rate and weight decay were all within noise. Dropout 0.0 / 0.05 / 0.10 (#10, #7, #8) span 0.5718 to 0.5925; LR 1.5e-4 (#6) and weight decay 0.03 (#12) landed at 0.5479 and 0.5642.
Trial 1 errored on the evaluation path, not on its configuration. Its training finished, but evaluation fell back from vLLM (the tensor-parallel degree of 6 does not divide the 16 attention heads) to Hugging Face generation and exceeded the platform's 3-hour experiment cap. The eval path was fixed before trial 2 (merge the adapter, serve on vLLM with tensor-parallel 4), which is why every later evaluation took under six minutes. max_seq_len 4096 was never scored.

What it did not establish: the winning margin. Trial 11 is a re-run of trial 7's exact configuration and scored 0.553836 against 0.592465, a spread of 0.039 with nothing but initialisation and data order changed (seeds were not pinned). Most differences in the table are smaller than that. Read 0.592 as the high draw of a configuration whose expected score is in the mid-0.5s, and treat any two trials within about 0.04 of each other as tied.

Search space. Five knobs were varied (LORA_R, LORA_DROPOUT, LEARNING_RATE, EPOCHS, WEIGHT_DECAY) plus the single MAX_SEQ_LEN probe. LR_SCHEDULER, GRAD_ACCUM, WARMUP_RATIO, LORAPLUS_LR_RATIO, USE_RSLORA, USE_DORA, BATCH_SIZE and the training method were never moved.

Observed training metrics (this checkpoint).

Table

Final train loss (mean over the run)	0.6348355285664822
Last logged train loss (step 560)	0.4464 (grad norm 0.28, token accuracy 0.882)
Final eval loss (teacher-forced, answer tokens)	0.5039476752281189
Eval mean token accuracy	0.8734
Train runtime	6,232.9755 s
Total FLOPs	8.461752173519176e+17
Throughput	2.163 samples/s, 0.09 steps/s

564 optimizer steps ran. The logged train loss fell from 5.9986 at step 10 (grad norm 7.09) to 0.4464 at step 560, with a minimum of 0.4230; the reported train loss is the mean over the run, not a converged value.

Evaluation

Protocol. All 375 validation rows, greedy decoding (temperature 0), max_new_tokens=2048, prompts rendered through the chat template. Because vLLM cannot attach a LoRA to this architecture, the platform merged the adapter into the base and generated with vLLM 0.19.1 at tensor-parallel 4 on four H100s; the pass took 284 s. The merged weights it served are the ones at this repository's root. The generation evaluator then scored every output against the reference rule as text.

Table with columns: Metric, Value
Metric	Value
ROUGE-L F-measure, mean over rows (the search metric)	0.592465
BLEU, mean over rows	0.433607
Exact match	0.0 (0 / 375)

What the text metrics miss, measured after the fact. The same 375 predictions (adapter/predictions.jsonl) were checked with pySigma 1.5.1 and pysigma-backend-splunk 2.1.0, which are not part of the platform's evaluator:

Table with columns: Check, Count, Rate
Check	Count	Rate
Output is a YAML mapping	347 / 375	92.5%
Parses as a Sigma rule (`SigmaRule.from_yaml`)	260 / 375	69.3%
Compiles to Splunk SPL	260 / 375	69.3%
`logsource` block equals the reference's	213 / 375	56.8%
Runaway generation (at least twice the reference length and over 2,000 characters)	95 / 375

Of the 115 parse failures, 74 are SigmaConditionError (the condition references a selection that was never defined or is malformed), 24 are YAML scanner errors, 11 other YAML errors, 3 parser errors; the rest are single cases. The runaway outputs are repetition loops in long list values: the median prediction is 659 characters against a reference median of 679, but the 90th percentile is 5,081 characters and the longest 10,462. They inflate nothing (ROUGE-L is recall-bounded by the reference) but they cost the 2048-token budget on a quarter of the rows and are the first thing to fix.

Exact match is zero by construction. The reference title is the SigmaHQ author's wording ("Turla Group Lateral Movement"); the model writes its own ("Turla Lateral Movement"). ROUGE-L gives partial credit for that; exact match gives none.

Baseline for comparison. Not measured. The untuned google/gemma-4-26B-A4B-it was never scored on these 375 rows, so nothing here quantifies how much of the score the fine-tuning is responsible for. This is the most important gap in this card.

Published comparison points, different task framing. Two small public fine-tunes report compile-style metrics on their own held-out sets: e12ex2/Qwen3-1.7B-SigmaRL (1.7B, 3,116 SigmaHQ pairs) reports 45.8% valid-and-Splunk-compilable on 24 prompts, and alirezaaminzadeh/sigmaforge-rule-generator (1.5B) reports 83% Splunk compilation on 60 prompts whose input includes the rule's own description, log source and ATT&CK tags. This model's 69.3% sits between them on a prompt that withholds the title and detection; none of the three measures whether a compiled rule matches the right events.

This is a validation split the search selected against. 11 trials were scored on these same 375 rows and the best was kept, so expect optimistic bias on top of the re-run spread already described. The rows are drawn from the same SigmaHQ snapshot as training, so they are unseen rules, not rules written after the training data.

The evaluation set is reproducible. adapter/predictions.jsonl holds every one of the 375 rows: instruction, requirement block, prediction and gold. The builder's manifest (SigmaHQ commit, seed, drop counts, prompt field counts) is summarised under Training details.

Limitations and bias

One number, wide error bars. The same configuration scored 0.592 and 0.554 in two runs. Anyone deploying this should re-evaluate on their own requirements rather than trust either figure.

No baseline, so no established gain. See Evaluation.

A third of outputs are not valid Sigma. 31% fail to parse, most often because the condition line names a selection the rule never defined. Validate every output with pySigma before it goes anywhere near a SIEM.

A quarter of outputs never stop. Repetition loops in long lists consume the whole token budget. Cap max_new_tokens at about 1,024 and treat a truncated output as a failure.

Wording overlap is not detection quality. ROUGE-L rewards reproducing the reference's phrasing. A rule with the correct logic in different field order scores low; a rule that copies most of the reference but breaks one condition scores high. No metric here runs the rule against events.

Prompt shape is the contract. Change the system prompt, the instruction sentence, the Requirement: line format, or the code fence, and you are evaluating a model nobody measured.

Domain narrowness. SigmaHQ's coverage: mostly Windows process creation, plus Linux, macOS, cloud and network sources in proportion to that repository. Log sources absent from SigmaHQ, and non-English requirements, are unmeasured.

Inherits all biases and limitations of the base model. This adapter changes 0.14% of the parameters and was not evaluated for safety or fairness. The base model's own card governs those properties.

Merged-weights equivalence

The root weights are adapter/ merged into the base in bfloat16 (PeftModel.merge_and_unload through autocatalyst.cli.merge_lora, transformers 5.6.2), re-sharded to 5 GB safetensors with transformers 5.7.0.dev0. They are the exact directory the evaluation above was served from, so the numbers in this card are the fused model's numbers. A bfloat16 merge of a float32 adapter rounds the update once; no difference was measured, and none is expected at this magnitude. The merged directory keeps the untouched vision tower and the processor config so it loads as the base does.

Environmental impact

Table

Hardware	6x NVIDIA H100 80GB HBM3
Training time	103.9 minutes (6,232.98 s)
Cloud provider / region	on-premise

Covers this trial only. The full 12-trial search that selected it took 22 h 21 m on the same hardware, three hours of which were trial 1's errored evaluation.

Framework versions

PEFT 0.18.1
TRL: 1.0.0
Transformers: 5.7.0.dev0 (git main) for training and re-sharding; 5.6.2 for the merge
Pytorch: 2.5.1+cu121
bitsandbytes: 0.49.2
flash-attn: 2.8.3
vLLM: 0.19.1 (evaluation)
Python: 3.12.3

PEFT's version is the one recorded in adapter/adapter_config.json at save time; the rest are the pinned versions of the training and evaluation environments. transformers is a git-main build: the gemma4 architecture is not in the stable PyPI release used for training.

Citation

bibtex

@misc{gemma4_sigma_rules_2026,  title  = {Gemma-4-26B-A4B Sigma Rule Generator},  author = {Banerjee, Aaron and Anbuselvan, Pooja and Jodhpurkar, Om},  year   = {2026},  url    = {https://huggingface.co/SASVAAI/Gemma-4-26B-A4B-sigma-rules}}

Training rules: SigmaHQ contributors, https://github.com/SigmaHQ/sigma, Detection Rule License 1.1.

Model details

Table

Developed by	SASVA AI Model Cognition Labs (MCL) Team
Base model	`google/gemma-4-26B-A4B-it`
Base revision	`4d7ae4984b7db7de8f8457170b3f1a419ee76d52`
Base parameters	25,805,936,206 total (Hub safetensors metadata); mixture-of-experts, ~4B active per token
Architecture family	`gemma4` (`Gemma4ForConditionalGeneration`; text tower with 30 layers, 128 experts, hidden size 2816, vocabulary 262,144)
Adaptation	LoRA (`r=32`, `alpha=64`, `dropout=0.05`, rsLoRA off, DoRA off), merged into the root weights; adapter kept under `adapter/`
Trainable modules	`q_proj`, `k_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` on all 30 layers; `v_proj` on the 25 sliding-window layers (see below)
Excluded modules	`.vision_tower.` (the vision tower is untouched; training and evaluation were text-only)
Training method	`qlora` (`--load-in-4bit`, run 8 / trial 7)
Refinement	none
Precision	training: 4-bit NF4 base with double quantisation, bf16 compute, adapter in float32; root weights: bfloat16 merge
Language	English
License	Apache-2.0 (inherited from the base model); training rules are DRL 1.1

Intended use

The model was trained on a specific prompt shape and that shape is part of the contract:

System prompt (verbatim): "You are a Sigma rule generator. Given a plain-language detection requirement, output a valid Sigma detection rule in YAML format. Start with title: and include all standard Sigma fields (title, status, description, logsource, detection, level, and any relevant fields/tags). Output only the raw YAML with no code fences, no explanations, and no additional text."
User turn: the instruction "You are a detection engineer. Write a valid Sigma rule (YAML) that satisfies the requirement. Output only the YAML.", a blank line, then the requirement block inside a bare ``` fence. This is the exact string the evaluator rendered (instruction + "\n\n```\n" + requirement_block + "\n```").
The requirement block is one to four lines, in this order and with this wording: an optional Log source: <product> / <category>. line, the mandatory Requirement: <description> line, an optional ATT&CK: T1059.003, T1218.011. line, and an optional Known false positives: <a>; <b>. line. In training, each optional line was present with probability 0.75 / 0.6 / 0.5 respectively, so the model handles both terse and detailed requests.
Applied through the tokenizer's chat template (chat_template.jinja, shipped in this repo) with . Do not concatenate strings by hand.

Out-of-scope use. Deploying an unreviewed rule to a SIEM. Generating rules for log sources absent from SigmaHQ. Any use as a detector of the threats the rules describe.

How to get started

The prompt pieces are the same in every path. Verbatim from the training invocation; do not paraphrase.

python

SYSTEM = (    "You are a Sigma rule generator. Given a plain-language detection requirement, "    "output a valid Sigma detection rule in YAML format. Start with `title:` and "    "include all standard Sigma fields (title, status, description, logsource, "    "detection, level, and any relevant fields/tags). Output only the raw YAML with "    "no code fences, no explanations, and no additional text.")INSTRUCTION = (    "You are a detection engineer. Write a valid Sigma rule (YAML) that satisfies "    "the requirement. Output only the YAML.")requirement = (    "Log source: windows / process_creation.\n"    "Requirement: Detects the execution of the hacktool Rubeus via PE information "    "or command line parameters\n"    "ATT&CK: T1003, T1558.003.")messages = [    {"role": "system", "content": SYSTEM},    {"role": "user", "content": f"{INSTRUCTION}\n\n```\n{requirement}\n```"},]

1. Fused weights with transformers (about 52 GB of accelerator memory in bfloat16; this is the exact directory the evaluation was served from):

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer REPO = "SASVAAI/Gemma-4-26B-A4B-sigma-rules"# AutoModelForCausalLM resolves to Gemma4ForConditionalGeneration on# transformers 5.7 and loads the text model.tokenizer = AutoTokenizer.from_pretrained(REPO)model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")model.eval() inputs = tokenizer.apply_chat_template(    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True).strip())

2. Fused weights with vLLM (how the numbers below were produced; the tensor-parallel degree must divide the 16 attention heads):

bash

pip install "vllm>=0.19.1"vllm serve SASVAAI/Gemma-4-26B-A4B-sigma-rules --tensor-parallel-size 4 --max-model-len 4096

pip install "vllm>=0.19.1"vllm serve SASVAAI/Gemma-4-26B-A4B-sigma-rules --tensor-parallel-size 4 --max-model-len 4096

python

from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")r = client.chat.completions.create(    model="SASVAAI/Gemma-4-26B-A4B-sigma-rules",    messages=messages, temperature=0, max_tokens=1024,)print(r.choices[0].message.content)

3. Adapter on the base model (base in 4-bit fits one 24 GB-class GPU, about 16 GB; verified on CPU in bf16 with these exact calls):

python

import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer BASE = "google/gemma-4-26B-A4B-it"REPO = "SASVAAI/Gemma-4-26B-A4B-sigma-rules"tokenizer = AutoTokenizer.from_pretrained(REPO, subfolder="adapter")model = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, REPO, subfolder="adapter")model.eval()# then generate exactly as in path 1

For the 4-bit base pass

text

quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)

quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)

Actual output of path 3 for the prompt above (CPU, bf16, greedy), first lines:

text

title: HackTool - Rubeus Executiondescription: Detects the execution of the hacktool Rubeus via PE information or command line parameterslogsource:  category: process_creation  product: windowsdetection:  selection_img:    OriginalFileName: Rubeus.exe  selection_cli:    CommandLine|contains:    - ' /ticket'    - ' /ptt'    - ' /asktgt'    - ' /askns'    - ' /ptt'    - ' /ptt'      <- the repetition loop described under Evaluation;    ...               this prompt is one of the runaway cases.

title: HackTool - Rubeus Executiondescription: Detects the execution of the hacktool Rubeus via PE information or command line parameterslogsource:  category: process_creation  product: windowsdetection:  selection_img:    OriginalFileName: Rubeus.exe  selection_cli:    CommandLine|contains:    - ' /ticket'    - ' /ptt'    - ' /asktgt'    - ' /askns'    - ' /ptt'    - ' /ptt'      <- the repetition loop described under Evaluation;    ...               this prompt is one of the runaway cases.

Decoding matters. The metric was scored greedily with max_new_tokens=2048 through the chat template. No sampling setting was validated.

Training details

Row construction, as recorded in the builder's manifest:

Source directories rules/, rules-threat-hunting/ and rules-emerging-threats/; rules-compliance/ skipped. 3,757 rules read, 11 dropped for exceeding 8,000 characters, 3,746 kept.
Input carries the rule's description as the requirement and, per rule and deterministically from the seed, sometimes the log source (75%), the ATT&CK technique ids parsed from tags (60%) and the falsepositives list (50%). It never contains the rule's title or detection block, which are the answer. In the 375 validation rows: 282 carry a log source, 206 an ATT&CK line, 79 a false-positives line.
Output is the rule re-serialised with only title, description, logsource, , , , , in that order. , , , , , , and are dropped as unlearnable noise.

Table

Train samples	3,371 rules
Validation samples	375 rules (10% of groups)
Group overlap	0 groups
Prompt format	chat template + system prompt + instruction / fenced-requirement user turn (see Intended use)
Loss masking	answer tokens only; prompt tokens set to `-100`
Truncation	sequences cut to `max_seq_len` 2048; a 7.5 kB rule is about 2,000 tokens, so the longest targets lose their tail during training

Method

Table

SFT method	`qlora`
Base quantisation during training	4-bit NF4, double quantisation, bf16 compute (`bitsandbytes`)
Refinement stage	none
Auto class	`AutoModelForCausalLM` (resolves to `Gemma4ForConditionalGeneration`)
Hardware	6x NVIDIA H100 80GB HBM3, `torchrun --nproc_per_node=6`

The project allowed one method (qlora). No refinement stage ran; the published adapter is the SFT adapter and the root weights are its merge.

Final hyperparameters

Table with columns: Hyperparameter, Value, Source
Hyperparameter	Value	Source
`learning_rate`	0.0002	`[TRAIN]` cmdline
`lr_scheduler_type`	cosine	`[TRAIN]` cmdline
`num_train_epochs`	4	`[TRAIN]` cmdline, `adapter/trainer_state.json`

Effective batch size: 24 (1 x 4 x 6). Optimizer steps: 564 (141 per epoch).

Provenance note: every value above was recovered from the platform database (runs, experiments, events tables for run 8) and cross-checked against the literal [TRAIN] command line recorded in the run log and against the shipped adapter/adapter_config.json.

How these values were chosen

These hyperparameters were selected by an automated search (autocatalyst.cli.run_autoresearch): an agent proposes one change at a time, runs train then eval, and keeps or discards on rouge_l (higher is better).

Table with columns: #, r, Dropout, LR, Epochs, Seq len, Weight decay, Train, Eval, rouge_l, Kept
#	r	Dropout	LR	Epochs	Seq len	Weight decay	Train	Eval	rouge_l	Kept
1	16	0.05	2e-4	2	4096	0.01	3 h 00 m	–	error

All trials: QLoRA, lora_alpha = 2 x r, cosine schedule, batch 1 per device, gradient accumulation 4, warmup 0.05, LoRA+ off, rsLoRA and DoRA off.

What the search actually established.

Epochs mattered up to 4. With everything else fixed at r=16, 2 → 3 → 4 epochs moved ROUGE-L 0.5238 → 0.5427 → 0.5767 (#2, #3, #4); 5 epochs fell back to 0.5427 (#5).
Rank 16 → 32 helped; 64 did not help further. #4 → #7 moved 0.5767 → 0.5925; #9 at r=64 scored 0.5888, inside the re-run spread below.
Dropout, learning rate and weight decay were all within noise. Dropout 0.0 / 0.05 / 0.10 (#10, #7, #8) span 0.5718 to 0.5925; LR 1.5e-4 (#6) and weight decay 0.03 (#12) landed at 0.5479 and 0.5642.
Trial 1 errored on the evaluation path, not on its configuration. Its training finished, but evaluation fell back from vLLM (the tensor-parallel degree of 6 does not divide the 16 attention heads) to Hugging Face generation and exceeded the platform's 3-hour experiment cap. The eval path was fixed before trial 2 (merge the adapter, serve on vLLM with tensor-parallel 4), which is why every later evaluation took under six minutes. max_seq_len 4096 was never scored.

Observed training metrics (this checkpoint).

Table

Final train loss (mean over the run)	0.6348355285664822
Last logged train loss (step 560)	0.4464 (grad norm 0.28, token accuracy 0.882)
Final eval loss (teacher-forced, answer tokens)	0.5039476752281189
Eval mean token accuracy	0.8734
Train runtime	6,232.9755 s
Total FLOPs	8.461752173519176e+17
Throughput	2.163 samples/s, 0.09 steps/s

Evaluation

Table with columns: Metric, Value
Metric	Value
ROUGE-L F-measure, mean over rows (the search metric)	0.592465
BLEU, mean over rows	0.433607
Exact match	0.0 (0 / 375)

Table with columns: Check, Count, Rate
Check	Count	Rate
Output is a YAML mapping	347 / 375	92.5%
Parses as a Sigma rule (`SigmaRule.from_yaml`)	260 / 375	69.3%
Compiles to Splunk SPL	260 / 375	69.3%
`logsource` block equals the reference's	213 / 375	56.8%
Runaway generation (at least twice the reference length and over 2,000 characters)	95 / 375

Limitations and bias

One number, wide error bars. The same configuration scored 0.592 and 0.554 in two runs. Anyone deploying this should re-evaluate on their own requirements rather than trust either figure.

No baseline, so no established gain. See Evaluation.

A quarter of outputs never stop. Repetition loops in long lists consume the whole token budget. Cap max_new_tokens at about 1,024 and treat a truncated output as a failure.

Prompt shape is the contract. Change the system prompt, the instruction sentence, the Requirement: line format, or the code fence, and you are evaluating a model nobody measured.

Merged-weights equivalence

Environmental impact

Table

Hardware	6x NVIDIA H100 80GB HBM3
Training time	103.9 minutes (6,232.98 s)
Cloud provider / region	on-premise

Covers this trial only. The full 12-trial search that selected it took 22 h 21 m on the same hardware, three hours of which were trial 1's errored evaluation.

Framework versions

PEFT 0.18.1
TRL: 1.0.0
Transformers: 5.7.0.dev0 (git main) for training and re-sharding; 5.6.2 for the merge
Pytorch: 2.5.1+cu121
bitsandbytes: 0.49.2
flash-attn: 2.8.3
vLLM: 0.19.1 (evaluation)
Python: 3.12.3

Citation

bibtex

@misc{gemma4_sigma_rules_2026,  title  = {Gemma-4-26B-A4B Sigma Rule Generator},  author = {Banerjee, Aaron and Anbuselvan, Pooja and Jodhpurkar, Om},  year   = {2026},  url    = {https://huggingface.co/SASVAAI/Gemma-4-26B-A4B-sigma-rules}}

Training rules: SigmaHQ contributors, https://github.com/SigmaHQ/sigma, Detection Rule License 1.1.

Gemma-4-26B-A4B-sigma-rules

README

Model details

Intended use

How to get started

Training details

Method

Final hyperparameters

How these values were chosen

Evaluation

Limitations and bias

Merged-weights equivalence

Environmental impact

Framework versions

Citation

Explore FriendliAI today

README

Model details

Intended use

How to get started

Training details

Method

Final hyperparameters

How these values were chosen

Evaluation

Limitations and bias

Merged-weights equivalence

Environmental impact

Framework versions

Citation