otherwhere1

Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA-mlx-8Bit

README

License: apache-2.0

Use with mlx

bash

pip install mlx-lm

python

from mlx_lm import load, generate model, tokenizer = load("otherwhere1/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA-mlx-8Bit") prompt="hello" if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:    messages = [{"role": "user", "content": prompt}]    prompt = tokenizer.apply_chat_template(        messages, tokenize=False, add_generation_prompt=True    ) response = generate(model, tokenizer, prompt=prompt, verbose=True)

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

otherwhere1

Model Tree

Base

Naphula/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA

Quantized

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Use with mlx

bash

pip install mlx-lm

pip install mlx-lm

python

from mlx_lm import load, generate model, tokenizer = load("otherwhere1/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA-mlx-8Bit") prompt="hello" if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:    messages = [{"role": "user", "content": prompt}]    prompt = tokenizer.apply_chat_template(        messages, tokenize=False, add_generation_prompt=True    ) response = generate(model, tokenizer, prompt=prompt, verbose=True)

from mlx_lm import load, generate model, tokenizer = load("otherwhere1/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA-mlx-8Bit") prompt="hello" if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:    messages = [{"role": "user", "content": prompt}]    prompt = tokenizer.apply_chat_template(        messages, tokenize=False, add_generation_prompt=True    ) response = generate(model, tokenizer, prompt=prompt, verbose=True)