neurocti-qwen3-32b-orion10k-instruct-fb16-r32-lr0.0001-sl8192-e3-v1 API & Inference Endpoint

Model description

The NeuroCTI Cybersecurity Cyber Threat Intelligence (CTI) model was fine-tuned on the orion10k dataset. The purpose of this model is to help in automatically generating YARA detection rules from unstructured CTI reports.

Intended uses & limitations

Experimental use for now

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 2
total_train_batch_size: 8
total_eval_batch_size: 4
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 3.0

Training results

Table with columns: Training Loss, Epoch, Step, Validation Loss
Training Loss	Epoch	Step	Validation Loss
1.6419	0.1667	1	6.5215
1.8202	0.3333	2	6.5088
1.6672	0.6667	4	6.2647
1.6093	1.0	6	5.1988
1.5079

Framework versions

PEFT 0.15.2
Transformers 4.51.3
Pytorch 2.6.0+cu124
Datasets 3.5.1
Tokenizers 0.21.1

Model description

Intended uses & limitations

Experimental use for now

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 2
total_train_batch_size: 8
total_eval_batch_size: 4
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 3.0

Training results

Table with columns: Training Loss, Epoch, Step, Validation Loss
Training Loss	Epoch	Step	Validation Loss
1.6419	0.1667	1	6.5215
1.8202	0.3333	2	6.5088
1.6672	0.6667	4	6.2647
1.6093	1.0	6	5.1988
1.5079

Framework versions

PEFT 0.15.2
Transformers 4.51.3
Pytorch 2.6.0+cu124
Datasets 3.5.1
Tokenizers 0.21.1

neurocti-qwen3-32b-orion10k-instruct-fb16-r32-lr0.0001-sl8192-e3-v1

README

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Explore FriendliAI today

README

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions