Model details
Table with columns: Field, Value| Field | Value |
|---|
| Architecture | Qwen3ForCausalLM-compatible decoder-only Transformer |
| Parameters | 1,120,772,224 |
| Layers | 19 |
| Hidden / intermediate size | 1,920 / 7,680 |
| Attention heads / KV heads | 15 / 15 |
| Context | 256 tokens: one BOS token followed by up to 255 DNA bases |
| Checkpoint | Final m5.1 checkpoint, step 59,158 |
| Approximate token exposure | 166.0B nucleotide tokens over the inherited training lineage |
| Stored weight dtype | float32 |
| License | Apache-2.0 |
The source checkpoint is
gs://marin-us-east5/checkpoints/dna-bolinas-mix-v0.9-p1B-i24-exp135-zoonomia-m5.1-bef41e/hf/step-59158.
The commit-pinned training script defines the experiment.
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "bolinas-dna/marin-dna-exp135-m5.1"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
The float32 checkpoint is approximately 4.18 GiB. Convert or load it in a lower-precision dtype only if that is appropriate for your downstream use.
The bundled tokenizer is a case-insensitive, single-nucleotide tokenizer with seven tokens:
Table with columns: Token, ID| Token | ID |
|---|
[PAD] | 0 |
[UNK] | 1 |
[BOS] | 2 |
a | 3 |
c | 4 |
g | 5 |
Pass raw DNA strings containing A, C, G, and T without spaces or separators. The tokenizer lowercases input and automatically prepends [BOS]; the model has no EOS token and the tokenizer does not append one. Characters outside the four canonical bases map to [UNK]. Because BOS occupies one of the 256 model positions, inputs are limited to 255 DNA bases.
Training lineage and data
m5.1 is a continued-training lineage with approximately 166.0B total inherited and newly seen nucleotide tokens:
- approximately 42.0B inherited tokens from the pre-cooldown point of a uniform three-region mixture (CDS, upstream, and downstream);
- approximately 62.0B tokens from a continued uniform three-region mixture; and
- approximately 62.0B tokens from a uniform five-region mixture adding enhancer and ncRNA sequence.
The five-region inventory below describes the resources used across the m5.1 lineage. It does not mean that all five datasets contributed to every training phase.
Training datasets
Matched training-validation probes
These matched datasets were validation probes, not training data.
Intended uses
This checkpoint is intended for genomic language-model research, including sequence likelihood analyses, representation extraction, variant-effect research, and reproducible comparison with the experiments reported in the accompanying blog. It is a base model, not a task-specific predictor.
Evaluation and caveats
The downstream evaluation-only datasets are Mendelian variant effects and saturation genome editing (SGE). They were not used as training data. Mendelian and SGE use different construction, matching, and aggregation protocols, so their score levels must not be compared directly.
The supported interpretation is documented in the blog analysis hub and the collaborator-review dossier. In particular, the paired analysis does not support a claim that m5.1 significantly outperforms Evo 2 40B or AlphaGenome; those comparisons are statistical ties under the reported Mendelian analysis.
Additional limitations:
- The 255-base input window is short relative to many regulatory and structural genomic effects.
- The tokenizer represents only the four canonical DNA bases directly; other symbols become
[UNK].
- Performance depends on the downstream scoring or probing protocol and can vary substantially by variant category.
- This research checkpoint has not been validated for clinical diagnosis or medical decision-making.
Provenance and reproducibility
The tracked release manifest records source and destination inventories, byte sizes, checksums, model configuration, tokenizer details, dataset revisions, and the deterministic inference reference.