rudy-nemo-12b-v1-unquantized API & Inference Endpoint

Rudy-Nemo-12.2B-v1 (Unquantized)

This repository contains the raw, unquantized 16-bit (safetensors) weights for Rudy-Nemo-12B-v1, intended for further merging, fine-tuning, or high-precision inference.

Looking for ready-to-run GGUF files? > Check out the GGUF repository for LM Studio, Ollama, and llama.cpp:
RoseRudolph/rudy-nemo-12b-v1

This is a merge of pre-trained language models created using mergekit.

merge method

This model was merged using the SLERP merge method.

merge victims

The following models were included in the merge:

yaml config

The following YAML configuration was used to produce this model:

yaml

slices:  - sources:      - model: mistralai/Mistral-Nemo-Instruct-2407        layer_range: [0, 40]      - model: TheDrummer/Rocinante-12B-v1.1        layer_range: [0, 40] merge_method: slerpbase_model: mistralai/Mistral-Nemo-Instruct-2407tokenizer_source: base parameters:  t:    - filter: self_attn      value: [0.00, 0.25, 0.50, 0.75, 1.00]    - filter: mlp      value: [0.00, 0.25, 0.50, 0.75, 1.00]    - value: 0.50 dtype: bfloat16

Rudy-Nemo-12.2B-v1 (Unquantized)

This repository contains the raw, unquantized 16-bit (safetensors) weights for Rudy-Nemo-12B-v1, intended for further merging, fine-tuning, or high-precision inference.

Looking for ready-to-run GGUF files? > Check out the GGUF repository for LM Studio, Ollama, and llama.cpp:
RoseRudolph/rudy-nemo-12b-v1

This is a merge of pre-trained language models created using mergekit.

merge method

This model was merged using the SLERP merge method.

merge victims

The following models were included in the merge:

yaml config

The following YAML configuration was used to produce this model:

yaml

slices:  - sources:      - model: mistralai/Mistral-Nemo-Instruct-2407        layer_range: [0, 40]      - model: TheDrummer/Rocinante-12B-v1.1        layer_range: [0, 40] merge_method: slerpbase_model: mistralai/Mistral-Nemo-Instruct-2407tokenizer_source: base parameters:  t:    - filter: self_attn      value: [0.00, 0.25, 0.50, 0.75, 1.00]    - filter: mlp      value: [0.00, 0.25, 0.50, 0.75, 1.00]    - value: 0.50 dtype: bfloat16

rudy-nemo-12b-v1-unquantized

README

Rudy-Nemo-12.2B-v1 (Unquantized)

merge method

merge victims

yaml config

Explore FriendliAI today

README

Rudy-Nemo-12.2B-v1 (Unquantized)

merge method

merge victims

yaml config