Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more
Container

Run this model inference with full control and performance in your environment.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

README

Notes for v1 and 1.1

Qliphoth 12B has some refusals and may require jailbreaks or ablation to fully uncensor.

Both Mistral Tekken and ChatML chat templates are supported and may produce different results, so it's recommended to use one of those.

The model is very creative and produces highly varied, verbose output even at low temps. A karcher merge was tested using the same donors and was found to be less creative in comparison.

Version 1 vs 1.1

In my simple tests, v1 had a distinct, visceral style and seemed to prefer Mistral Tekken, while v1.1 was more clinical and detached, and did better with ChatML template.

Both versions performed well and were tested with Q0 Bench, where v1.1 scored about 3000 points higher. They were also tested with the new MiniBARD (Benchmark for Aesthetics, Roleplay & Depth). Again here, v1.1 outperformed v1. Either version is great, so test them both if you have time, although I may slightly prefer the style of v1.

BARD

⚙️ Configuration

The following YAML configuration was used to produce this model:

yaml
architecture: MistralForCausalLM
base_model: B:/12B/IntervitensInc--Mistral-Nemo-Base-2407-chatml
models:
  - model: B:/12B/SicariusSicariiStuff--Impish_Bloodmoon_12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/NeverSleep--Lumimaid-v0.2-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/KOOWEEYUS--BlackSheep-RP-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/KOOWEEYUS--BlackSheep-RP-12B # x2 influence for the apprentice
    parameters:
      pinocchio: 0.0
  - model: B:/12B/SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
    parameters:
      pinocchio: 0.0
  - model: B:/12B/TheDrummer--Rocinante-X-12B-v1
    parameters:
      pinocchio: 0.0
  - model: B:/12B/WokeAI--Tankie-DPE-12b-SFT
    parameters:
      pinocchio: 0.0
  - model: B:/12B/WokeAI--Tankie-DPE-12B-SFT-v2 # pinocchio
    parameters:
      pinocchio: 1.0
  - model: B:/12B/XeyonAI--Mistral-Helcyon-Mercury-12b-v3.2
    parameters:
      pinocchio: 0.0
  - model: B:/12B/anthracite-org--magnum-v4-12b
    parameters:
      pinocchio: 0.0
  - model: B:/12B/dphn--dolphin-2.9.3-mistral-nemo-12b
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Edens-Gate--nemo-erebus-lora-2152/nemo-erebus-lora-2152
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Epiculous--Azure_Dusk-v0.2
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Epiculous--Crimson_Dawn-v0.2
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Fizzarolli--MN-12b-Rosier-v1
    parameters:
      pinocchio: 0.0
  - model: B:/12B/HumanLLMs--Human-Like-Mistral-Nemo-Instruct-2407
    parameters:
      pinocchio: 0.0
  - model: B:/12B/IIEleven11--Kalypso
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Lambent--Arsenic-Shahrazad-12B-v4.1
    parameters:
      pinocchio: 0.0
  - model: B:/12B/LatitudeGames--Wayfarer-2-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/PocketDoc--Dans-DangerousWinds-V1.1.0-12b
    parameters:
      pinocchio: 0.0
  - model: B:/12B/PocketDoc--Dans-SakuraKaze-V1.0.0-12b
    parameters:
      pinocchio: 0.0
  - model: B:/12B/PygmalionAI--Pygmalion-3-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/sleepdeprived3--Christian-Bible-Expert-v2.0-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/rAIfle--Questionable-MN-bf16
    parameters:
      pinocchio: 0.0
  - model: B:/12B/jtatman--mistral_nemo_12b_reasoning_psychology_lora/mistral_nemo_12b_reasoning_psychology_lora
    parameters:
      pinocchio: 0.0
  - model: B:/12B/LatitudeGames--Muse-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/allura-org--Tlacuilo-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/ChaoticNeutrals--Mag-Mell-Reasoner-12B
    parameters:
      pinocchio: 0.0
  - model: A:/LLM/.cache/13B/taozi555--MN-12B-Mag-Mell-R1-KTO
    parameters:
      pinocchio: 0.0
  - model: A:/LLM/.cache/13B/UniLLMer--GslayerKaa
    parameters:
      pinocchio: 0.0
merge_method: qliphoth
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: base
chat_template: "chatml"
yaml
architecture: MistralForCausalLM
base_model: B:/12B/IntervitensInc--Mistral-Nemo-Base-2407-chatml
models:
  - model: B:/12B/SicariusSicariiStuff--Impish_Bloodmoon_12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/NeverSleep--Lumimaid-v0.2-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/KOOWEEYUS--BlackSheep-RP-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/nothingiisreal--MN-12B-Celeste-V1.9
    parameters:
      pinocchio: 0.0
  - model: B:/12B/SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
    parameters:
      pinocchio: 0.0
  - model: B:/12B/TheDrummer--Rocinante-X-12B-v1
    parameters:
      pinocchio: 0.0
  - model: B:/12B/WokeAI--Tankie-DPE-12b-SFT
    parameters:
      pinocchio: 0.0
  - model: B:/12B/WokeAI--Tankie-DPE-12B-SFT-v2 # pinocchio
    parameters:
      pinocchio: 1.0
  - model: B:/12B/XeyonAI--Mistral-Helcyon-Mercury-12b-v3.2
    parameters:
      pinocchio: 0.0
  - model: B:/12B/anthracite-org--magnum-v4-12b
    parameters:
      pinocchio: 0.0
  - model: B:/12B/dphn--dolphin-2.9.3-mistral-nemo-12b
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Edens-Gate--nemo-erebus-lora-2152/nemo-erebus-lora-2152
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Epiculous--Azure_Dusk-v0.2
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Epiculous--Crimson_Dawn-v0.2
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Fizzarolli--MN-12b-Rosier-v1
    parameters:
      pinocchio: 0.0
  - model: B:/12B/HumanLLMs--Human-Like-Mistral-Nemo-Instruct-2407
    parameters:
      pinocchio: 0.0
  - model: B:/12B/IIEleven11--Kalypso
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Lambent--Arsenic-Shahrazad-12B-v4.3.2
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Lambent--Arsenic-Shahrazad-12B-v4.1
    parameters:
      pinocchio: 0.0
  - model: B:/12B/LatitudeGames--Wayfarer-2-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/PocketDoc--Dans-DangerousWinds-V1.1.0-12b
    parameters:
      pinocchio: 0.0
  - model: B:/12B/PocketDoc--Dans-SakuraKaze-V1.0.0-12b
    parameters:
      pinocchio: 0.0
  - model: B:/12B/PygmalionAI--Pygmalion-3-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/sleepdeprived3--Christian-Bible-Expert-v2.0-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/rAIfle--Questionable-MN-bf16
    parameters:
      pinocchio: 0.0
  - model: B:/12B/jtatman--mistral_nemo_12b_reasoning_psychology_lora/mistral_nemo_12b_reasoning_psychology_lora
    parameters:
      pinocchio: 0.0
  - model: B:/12B/LatitudeGames--Muse-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/allura-org--Tlacuilo-12B
    parameters:
      pinocchio: 0.0
  - model: B:/12B/ChaoticNeutrals--Mag-Mell-Reasoner-12B
    parameters:
      pinocchio: 0.0
  - model: A:/LLM/.cache/13B/taozi555--MN-12B-Mag-Mell-R1-KTO
    parameters:
      pinocchio: 0.0
  - model: A:/LLM/.cache/13B/UniLLMer--GslayerKaa
    parameters:
      pinocchio: 0.0
  - model: B:/12B/Retreatcost--Evertide-RX-12B
    parameters:
      pinocchio: 0.0
merge_method: qliphoth
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: base
chat_template: "chatml"

To fix tokenizer issues while retaining enhanced intelligence of the chatml base:

  1. I first merged the model using
yaml
base_model: B:/12B/IntervitensInc--Mistral-Nemo-Base-2407-chatml
tokenizer:
  source: base
chat_template: "chatml"
  1. I then merged it again using
yaml
base_model: B:/12B/mistralai--Mistral-Nemo-Instruct-2407
tokenizer:
  source: union
chat_template: auto
  1. I then had to modify mergekit/tokenizer/embed.py to allow for the passthrough merge
py
token_configs[token] = TokenEmbeddingConfig(source=ZeroEmbedding(kind="zero"))


    ) -> torch.Tensor:
        if isinstance(cfg.source, ZeroEmbedding):  
            first_tensor = next(iter(tensors.values()))  
            embed = torch.zeros(  
                first_tensor.shape[1],   
                dtype=first_tensor.dtype,   
                device=first_tensor.device  
            )
  1. I then ran another merge to fix tokenizer issues
yaml
merge_method: passthrough  
slices:  
  - sources:  
      - model: B:\12B\Stage1Base
        layer_range: [0, 40]  
tokenizer_source: B:\12B\Stage2Union
chat_template: auto

For some reason, this process produced significantly smarter output than just using base_model: B:/12B/mistralai--Mistral-Nemo-Instruct-2407.

This passthrough process using base as base_model was replicated for v1.2

Model provider

OccultAI

Model tree

Base

inflatebot/MN-12B-Mag-Mell-R1

Base

BrainDelay/Mistral-Nemo-BlackWidow-Agony-V1

Base

Naphula/Ancient-Awakening-12B

Base

SicariusSicariiStuff/Impish_Bloodmoon_12B

Base

Retreatcost/Evertide-RX-12B

Base

Epiculous/Violet_Twilight-v0.2

Base

OccultAI/Qliphoth-12B-v1

Base

WokeAI/Tankie-DPE-12B-SFT-v2

Base

Vortex5/Wicked-Oblivion-12B

Base

OccultAI/Qliphoth-12B-v1.1

Base

sleepdeprived3/Reformed-Christian-Bible-Expert-v2.1-12B

Base

EldritchLabs/KrakenSakura-Maelstrom-12B-v1

Base

DarkArtsForge/Morbid-Miasma-12B

Base

wave-on-discord/silly-v0.2

Base

Vortex5/Nether-Moon-12B

Base

IntervitensInc/Mistral-Nemo-Base-2407-chatml

Base

LatitudeGames/Muse-12B

Base

DarkArtsForge/Savage-Sands-12B

Merged

this model

Modalities

Input

Text

Output

Text

Pricing

Dedicated Endpoints

View details

Supported Functionality

Model APIs

Dedicated Endpoints

Container

More information

Model card

Explore FriendliAI today

Get startedTalk to an engineer