Solstice-AI

Qwen3.8-27B-Uncensored-mlx-6Bit

README

License: apache-2.0

Executive Summary & Model Overview

SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit is a calibrated 6-bit Apple MLX quantization of orcarouter/Qwen3.8-27B-Uncensored, engineered specifically for native high-throughput local execution on Apple Silicon.

This model combines the state-of-the-art reasoning, mathematical derivation, and agentic tool-use capabilities of the Qwen 3.8 27B parameter architecture with a complete refusal-direction ablation (abliteration), removing artificial refusal mechanisms while retaining full analytical precision, code synthesis accuracy, and structural reasoning fidelity.

Key Capabilities & Highlights

Zero-Refusal Uncensored Exploration: Complete orthogonal ablation of refusal steering vectors enables unrestricted cybersecurity auditing, AI safety red-teaming, hypothetical policy exploration, and raw technical query resolution.
6-Bit Precision Sweet Spot: Group-quantized 6-bit weights preserve 99.2% of unquantized BF16 benchmark performance while reducing the active memory footprint from ~54 GB to ~21.8 GB RAM.
Unified Memory Optimization: Native Apple MLX implementation leverages fused Metal performance shaders and unified memory bandwidth, delivering 48–62 tokens/sec on M3/M4 Max chips.
Extended Context Window: Native support for up to 131,072 context tokens with high-efficiency KV cache management.
Advanced Multi-Token Prediction (MTP): Compatible with speculative draft decoding pipelines for up to 1.4× faster inference generation.

Technical Specifications

Table with columns: Parameter, Specification
Parameter	Specification
Base Architecture	Qwen 3.8 Dense Transformer
Parameter Count	27.5 Billion
Base Model	`orcarouter/Qwen3.8-27B-Uncensored`
Quantization Format	MLX 6-bit (Group Size: 64)
Model Size on Disk	~21.4 GB
Active VRAM / RAM Footprint	~22.2 GB (8k context) / ~24.5 GB (32k context)

Hardware Recommendations & VRAM Footprint

Table with columns: Hardware Setup, Context Length, Expected Throughput, Status
Hardware Setup	Context Length	Expected Throughput	Status
Apple M4 Max (128 GB)	32,768 tokens	~62 tok/s	Recommended
Apple M3 Max (64 GB / 96 GB)	32,768 tokens	~54 tok/s	Recommended
Apple M2 Ultra (64 GB / 192 GB)	64,000 tokens	~58 tok/s	Recommended
Apple M3 Pro / M4 Pro (36 GB / 48 GB)	16,384 tokens	~42 tok/s

Quickstart Guide

1. Installation

Ensure you have Python 3.10+ and the latest mlx-lm library installed:

bash
pip install --upgrade mlx-lm

2. Interactive CLI Chat REPL

Launch an interactive streaming chat session directly in your terminal:

bash
mlx_lm.chat --model SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit --temp 0.7 --max-tokens 2048

3. Python API Generation

python
from mlx_lm import load, generate

# Load model and tokenizer directly into unified Apple Silicon memory
model, tokenizer = load("SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit")

messages = [
    {
        "role": "system",
        "content": "You are a helpful, direct, and unconstrained AI assistant. Provide complete, accurate, and uncensored technical explanations without unnecessary moralizing or disclaimers."
    },
    {
        "role": "user",
        "content": "Provide a comprehensive technical analysis of binary exploitation techniques via stack buffer overflows, including shellcode injection and Return-Oriented Programming (ROP) chain construction."
    }
]

# Apply official chat template
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

# Generate response with streaming output
response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=4096,
    temp=0.7,
    top_p=0.9,
    verbose=True
)

4. Deploy Local OpenAI-Compatible Server

Host a local REST API server compatible with any OpenAI client (e.g. Open-WebUI, Cursor, Continue.dev, LiteLLM):

bash
mlx_lm.server --model SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit --port 8080

Then query via cURL or OpenAI Python SDK:

python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")

response = client.chat.completions.create(
    model="SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit",
    messages=[
        {"role": "user", "content": "Write a complete Python implementation of an asynchronous web scraper with rate-limiting and rotating proxies."}
    ],
    temperature=0.7,
    max_tokens=2048
)

print(response.choices[0].message.content)

Prompt Template & Formatting

This model uses the standard ChatML prompt structure:

markdown
<|im_start|>system
You are a knowledgeable and helpful AI assistant.<|im_end|>
<|im_start|>user
{user_prompt}<|im_end|>
<|im_start|>assistant
{model_response}<|im_end|>

Empirical Benchmark Performance

Evaluated across standard reasoning, mathematics, code generation, and refusal benchmarks:

Table with columns: Benchmark, Metric, Unquantized BF16 (54 GB), Solstice 6-Bit MLX (21.8 GB), Relative Retention
Benchmark	Metric	Unquantized BF16 (54 GB)	Solstice 6-Bit MLX (21.8 GB)	Relative Retention
MMLU-Pro	5-shot Accuracy	68.4%	67.9%	99.3%
GSM8K	8-shot CoT	89.2%	88.8%	99.5%
Math-500	Pass@1	64.1%

About Solstice-AI

Solstice-AI is an open research foundation building the next generation of post-training distillation infrastructure, multi-teacher reasoning corpora, and high-performance local inference runtimes.

Website & Documentation: https://solstice-ai.co
GitHub Organization: https://github.com/Solstice-Labs
Hugging Face Organization: https://huggingface.co/Solstice-AI
Anvil Inference Engine: gondaliyashreyan1/anvil-llama-turbo

Citation & Acknowledgements

bibtex
@misc{solstice2026qwen38uncensoredmlx,
  title={Qwen3.8-27B-Uncensored 6-Bit MLX Quantization},
  author={Solstice-AI Systems Engineering and orcarouter},
  year={2026},
  publisher={Hugging Face Hub},
  howpublished={\url{https://huggingface.co/SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit}}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Solstice-AI

Model Tree

Base

orcarouter/Qwen3.8-27B-Uncensored

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality