rudy-nemo-12b-v1-unquantized API & Inference Endpoint

Rudy-Nemo-12.2B-v1 (Unquantized)

This repository contains the raw, unquantized 16-bit (safetensors) weights for Rudy-Nemo-12B-v1, intended for further merging, fine-tuning, or high-precision inference.

Looking for ready-to-run GGUF files? > Check out the GGUF repository for LM Studio, Ollama, and llama.cpp:
RoseRudolph/rudy-nemo-12b-v1

This is a merge of pre-trained language models created using mergekit.

merge method

This model was merged using the SLERP merge method.

merge victims

The following models were included in the merge:

Quantized Version

(https://huggingface.co/RoseRudolph/rudy-nemo-12b-v1)

yaml config

The following YAML configuration was used to produce this model:

yaml
slices:
  - sources:
      - model: mistralai/Mistral-Nemo-Instruct-2407
        layer_range: [0, 40]
      - model: TheDrummer/Rocinante-12B-v1.1
        layer_range: [0, 40]

merge_method: slerp
base_model: mistralai/Mistral-Nemo-Instruct-2407
tokenizer_source: base

parameters:
  t:
    - filter: self_attn
      value: [0.00, 0.25, 0.50, 0.75, 1.00]
    - filter: mlp
      value: [0.00, 0.25, 0.50, 0.75, 1.00]
    - value: 0.50

dtype: bfloat16

Rudy-Nemo-12.2B-v1 (Unquantized)

This repository contains the raw, unquantized 16-bit (safetensors) weights for Rudy-Nemo-12B-v1, intended for further merging, fine-tuning, or high-precision inference.

Looking for ready-to-run GGUF files? > Check out the GGUF repository for LM Studio, Ollama, and llama.cpp:
RoseRudolph/rudy-nemo-12b-v1

This is a merge of pre-trained language models created using mergekit.

merge method

This model was merged using the SLERP merge method.

merge victims

The following models were included in the merge:

Quantized Version

(https://huggingface.co/RoseRudolph/rudy-nemo-12b-v1)

yaml config

The following YAML configuration was used to produce this model:

yaml
slices:
  - sources:
      - model: mistralai/Mistral-Nemo-Instruct-2407
        layer_range: [0, 40]
      - model: TheDrummer/Rocinante-12B-v1.1
        layer_range: [0, 40]

merge_method: slerp
base_model: mistralai/Mistral-Nemo-Instruct-2407
tokenizer_source: base

parameters:
  t:
    - filter: self_attn
      value: [0.00, 0.25, 0.50, 0.75, 1.00]
    - filter: mlp
      value: [0.00, 0.25, 0.50, 0.75, 1.00]
    - value: 0.50

dtype: bfloat16

rudy-nemo-12b-v1-unquantized

README

Rudy-Nemo-12.2B-v1 (Unquantized)

merge method

merge victims

Quantized Version

yaml config

Explore FriendliAI today

README

Rudy-Nemo-12.2B-v1 (Unquantized)

merge method

merge victims

Quantized Version

yaml config