poolside

Laguna-M.1-base

Deploy Dedicated

README

License: apache-2.0

Highlights

Large sparse MoE: Laguna M.1 is a 70-layer MoE transformer with 225B total parameters and 23B activated parameters per token
High-capacity expert routing: After 3 dense SwiGLU layers, Laguna M.1 uses 67 sparse MoE layers with 256 experts, top-k=16 routing and auxiliary-loss-free load balancing
Global attention architecture: Laguna M.1 uses global attention across all layers with 64 Q-heads, 8 KV-heads and softplus attention output gating
Pre-instruct base checkpoint: Trained with pre-training only; intended as a starting point for further training rather than direct chat/agentic use
Apache 2.0 license: Use and modify freely for commercial and non-commercial purposes

Model overview

Stage: pre-training only (no post-training or reinforcement learning)
Number of parameters: 225B total with 23B activated per token
Optimizer: Muon
Layers: 70 layers with global attention
Experts: 256 experts with 1 shared expert; top-k=16 routing
Dense layers: first 3 layers are dense SwiGLU; remaining 67 layers are sparse MoE
Attention: 64 Q-heads, 8 KV-heads, head dimension 128, with softplus attention output gating
Positional encoding: RoPE with YaRN
Modality: text-to-text (completion)
Context window: 262,144 tokens

Usage

Laguna M.1-base is a text-completion model. It has no chat template, reasoning, or tool-calling support — serve it without the reasoning/tool-call parsers and prompt it with raw text.

vLLM

Laguna support is available in vLLM (v0.21.0 and later, vllm-project/vllm#41129).

shell
pip install 'vllm>=0.21.0'

vllm serve \
    --model poolside/Laguna-M.1-base \
    --served-model-name laguna-base

Query the completions endpoint from any OpenAI-compatible client:

python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

completion = client.completions.create(
    model="laguna-base",
    prompt="def fibonacci(n):\n",
    max_tokens=128,
    temperature=0.7,
)
print(completion.choices[0].text)

SGLang

Laguna M.1-base is supported in SGLang via sgl-project/sglang#28400. As a completion model, serve it without the reasoning/tool-call parsers. A full serving recipe will be added here.

Transformers

Laguna is supported in Transformers v5.7.0 and later (huggingface/transformers#45673).

[!NOTE] Laguna M.1-base is a 225B-parameter model; loading the BF16 checkpoint in Transformers requires substantial multi-GPU memory (device_map="auto" shards across available devices). For single-node serving, vLLM is recommended.

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "poolside/Laguna-M.1-base"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

inputs = tokenizer("def fibonacci(n):\n", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

This model is licensed under the Apache 2.0 License.

Intended and Responsible Use

Laguna M.1 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna M.1 is subject to the Apache 2.0 License, and should be used consistently with Poolside's Acceptable Use Policy. We advise against circumventing Laguna M.1 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.

Please report security vulnerabilities or safety concerns to security@poolside.ai.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider