Model description
Vir2vec is a viral genomic language model (gLM) designed to produce fixed-length, genome-level embeddings that can be fine-tuned across downstream tasks such as viral discrimination, host-range prediction, and variant typing. For more details and training scripts check GitHub
Intended use
Vir2vec embeddings are intended for tasks including (but not limited to):
- Virus vs non-virus genome/read discrimination
- DNA vs RNA virus classification
- Host-range prediction
- Intra-genus separation (e.g., HIV-1 vs HIV-2)
- Variant/subtype typing (e.g., SARS-CoV-2 lineages)
- Phenotypic signal detection (e.g., tissue tropism proxies)
Model sizes
All Vir2vec model scales produce fixed-length 4,096-dimensional genome-level embeddings via max-pooled vocabulary logits:
- 422M (
main branch)
- 138M (
revision="138M")
- 17M (
revision="17M")
How to use
Load from Hugging Face
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("pabloarozarenad/Vir2vec", trust_remote_code=True) # Add revision=138M or revision=17M to change model size. 422M is default.model = AutoModelForCausalLM.from_pretrained("pabloarozarenad/Vir2vec", trust_remote_code=True) # Add revision=138M or revision=17M to change model size. 422M is default.model.eval()
Compute embeddings
dna = "ACGTAGCATCGCGATGACTGCATCACT"inputs = tokenizer(dna, return_tensors="pt") with torch.no_grad(): # Forward pass to obtain vocabulary logits [batch, seq_len, 4096] outputs = model(**inputs) logits = outputs.logits # Max-pool logits over token positions -> 4,096-dimensional embedding embedding = torch.max(logits, dim=1).values[0] print(embedding.shape) # torch.Size([4096])
Access
Vir2vec can be loaded upon request, subject to providing an institutional email address, a brief description of the intended use, and the associated IRB protocol number. For more info contact pablo.arozarenadonel01@universitadipavia.it