GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
olka-fi
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
olka-fi
Model Tree
poolside/Laguna-S-2.1
Input Modalities
Output Modalities
Supported Functionality
| Component | Precision | Why |
|---|---|---|
Routed experts (model.layers.{1..47}.mlp.experts.*) | MXFP4 (4-bit) | ~96% of the weights — the only place worth the size win |
Attention (incl. g_proj output gating), shared experts, router gate, layer-0 dense MLP | BF16 | sensitive / runs on every token — kept lossless from the source |
| Embeddings, lm_head, norms | BF16 | unchanged |
We lead with deterministic, reproducible faithfulness metrics. (Task evals served
over vLLM with continuous batching are not bitwise-deterministic — the same prompt at
temperature=0 can yield different reductions depending on batch composition — so
task accuracies carry a little serving noise; we quote one full-test-set score, where
that noise is small.)
| Metric | Result | What it shows |
|---|---|---|
| Perplexity (wikitext-2 test, 309k tokens) | 9.82 | language modeling intact — a broken quant lands in the hundreds |
| Routed-expert SQNR | ≈ 19 dB | reconstruction error is just the unavoidable 4-bit rounding (MXFP4 vs the BF16 source) |
| GSM8K (full test, 1319 questions, temp 0) | 94.5% | end-to-end reasoning intact through the quantized experts |
Why this is enough to trust the checkpoint:
PPL script: evals/eval_ppl_wikitext2.py in the qstream repo.
Laguna is natively supported in vLLM ≥ 0.23, and this checkpoint uses the stock compressed-tensors MXFP4 path — no patches needed:
bash
vllm serve olka-fi/Laguna-S-2.1-MXFP4 --served-model-name laguna --max-model-len 60000
Keep --max-model-len bounded: only the 12 full-attention layers grow KV with
context (~48 KiB/token in BF16 KV), the other 36 layers hold a fixed 512-token
sliding window. The native 1M context needs ~46 GiB of KV per sequence.
Observed throughput: ≈600 tok/s aggregate generation at batch size 16 on a single RTX PRO 6000 Blackwell (96 GB) — the whole checkpoint plus KV fits on one card.
--kv-cache-dtype fp8 is a verified free lunch on this model: no calibration
or static k/v scales needed. Measured against BF16 KV: wikitext-2 PPL 9.812 vs
9.816, passkey retrieval 20/20 vs 20/20 at all depths in ~30k-token contexts.
(Laguna's per-head k_norm keeps keys inside e4m3 range at scale 1.0.) This
halves KV to ~24 KiB/token — double the context or concurrency.
On SM120 (RTX PRO 6000 / Blackwell workstation) three extra knobs are required:
bash
VLLM_ATTENTION_BACKEND=FLASHINFER \VLLM_FLASHINFER_WORKSPACE_BUFFER_SIZE=1073741824 \vllm serve olka-fi/Laguna-S-2.1-MXFP4 --served-model-name laguna --max-model-len 60000 \--kv-cache-dtype fp8 \--speculative-config '{"model":"olka-fi/Laguna-S-2.1-MXFP4-dspark","method":"dspark","num_speculative_tokens":4}' \--attention-config.disable_flashinfer_q_quantization=1
FlashInfer because FA2 can't read fp8 KV; disable_flashinfer_q_quantization
because vLLM otherwise also quantizes queries to fp8, which flashinfer's fa2
prefill template rejects outside the SM100 trtllm-gen path; the 1 GiB workspace
because the default 394 MiB overflows on Laguna's 72-head sliding-window layers.
bash
qstream-quantize \--model_dir <poolside/Laguna-S-2.1> \--output_dir ./Laguna-S-2.1-MXFP4 \--include_layers "mlp.experts." \--workers 8
Activation-aware (γ-weighted) MSE scale selection with per-block search over
candidate E8M0 exponents; the BF16 remainder is passed through untouched and the
mixed-precision config.json (ignore list + expert targets) is emitted for
stock vLLM loading. Built with qstream @9f8cd0c.
Inherits the OpenMDW-1.1 license from the base model. This is a derivative (quantized) work of poolside's Laguna S 2.1.
Laguna S 2.1 is a 118B total parameter Mixture-of-Experts model with 8B activated parameters per token, designed for agentic coding and long-horizon work. It sits between Laguna XS 2.1 (33B-A3B) and Laguna M.1 (225B-A23B) in the Laguna series and shares the family recipe: a token-choice router with softplus gating over 256 routed experts plus one shared expert, grouped-query attention, and interleaved full/sliding-window attention.
enable_thinking| Model | Size | Terminal-Bench 2.1 | SWE-bench Multilingual | SWE-Bench Pro (Public Dataset) | DeepSWE | SWE Atlas (Codebase QnA) | Toolathlon Verified |
|---|---|---|---|---|---|---|---|
| Laguna S 2.1 | 118B-A8B | 70.2% | 78.5% | 59.4% | 40.4% | 46.2% | 49.7% |
| Tencent Hy3 | 295B-A21B |
Benchmarks as of 21 July 2026. Laguna S 2.1 in bold; a dash (-) marks a benchmark a model was not evaluated on. Scores marked * are as reported by third parties: Terminal-Bench 2.1 and DeepSWE via Artificial Analysis, SWE Atlas via Scale AI's official leaderboard, and Toolathlon Verified via its official leaderboard. Full evaluation trajectories: trajectories.poolside.ai.
Laguna S 2.1 uses the same laguna architecture as Laguna XS 2.1, so the same
engine integrations apply (vLLM, SGLang, Transformers, TRT-LLM, llama.cpp). At 118B
parameters the BF16 checkpoint needs multiple GPUs (roughly 236GB of weights);
quantized variants reduce this substantially.
shell
vllm serve \--model poolside/Laguna-S-2.1 \--tensor-parallel-size 4 \--tool-call-parser poolside_v1 \--reasoning-parser poolside_v1 \--enable-auto-tool-choice \--served-model-name laguna \--default-chat-template-kwargs '{"enable_thinking": true}'
[!NOTE] Optional: speculative decoding with DFlash. Pair with the Laguna S 2.1 DFlash draft model by adding
--speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash","num_speculative_tokens":7,"method":"dflash"}'.
shell
python -m sglang.launch_server \--model-path poolside/Laguna-S-2.1 \--tp-size 4 \--reasoning-parser poolside_v1 \--tool-call-parser poolside_v1 \--trust-remote-code
shell
trtllm-serve poolside/Laguna-S-2.1 --trust-remote-code \--tool_parser poolside_v1 --reasoning_parser laguna
Note the flag names differ from vLLM's (--tool_parser, and the reasoning parser
is laguna, not poolside_v1).
GGUF conversions are available at
poolside/Laguna-S-2.1-GGUF.
Serve with poolside's llama.cpp fork, branch
laguna, which carries
full Laguna support including DFlash speculative decoding. (Base Laguna support
is also in upstream review:
ggml-org/llama.cpp#25165.)
shell
git clone --branch laguna https://github.com/poolsideai/llama.cppcd llama.cpp && cmake -B build && cmake --build build -j./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf --jinja --port 8000# with DFlash speculative decoding:./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf \-md laguna-s-2.1-DFlash-BF16.gguf \--spec-type draft-dflash --spec-draft-n-max 15 -fa on --jinja --port 8000
Laguna S 2.1 has native reasoning support and works best with preserved thinking:
keep reasoning_content from prior assistant messages in the message history.
The model will generally reason before calling tools and between tool calls, and
may stop reasoning in follow-up steps if prior thinking blocks are dropped.
Thinking is controlled per request via the chat template:
python
extra_body={"chat_template_kwargs": {"enable_thinking": False}}
or at the server level with
--default-chat-template-kwargs '{"enable_thinking": true}'. For agentic coding
use cases we recommend enabling thinking and preserving reasoning in the message
history.
This model is licensed under the OpenMDW-1.1 License.
Laguna S 2.1 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna S 2.1 is subject to the OpenMDW-1.1 License, and should be used consistently with Poolside's Acceptable Use Policy. We advise against circumventing Laguna S 2.1 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.
Please report security vulnerabilities or safety concerns to security@poolside.ai.
| 71.7% |
| 75.8% |
| 57.9% |
| - |
| - |
| - |
| Inkling | 975B-A41B | 63.8% | - | 54.3% | - | - | 45.5%* |
| Nemotron 3 Ultra | 550B-A55B | 56.4% | 67.7% | - | - | - | 34.3%* |
| DeepSeek-V4-Pro Max | 1.6T-A49B | 64.0%* | 76.2% | 55.4% | 9.0%* | 27.2%* | 55.9%* |
| Kimi K3 | 2800B-A50B | 88.3% | - | - | 69% | - | - |
| Qwen 3.7 Max | - | 74.5%* | 78.3% | 60.6% | - | - | - |
| Muse Spark 1.1 | - | 80% | - | 61.5% | 53.3% | 42.2%* | 75.6% |
| Claude Fable 5 | - | 88% | - | 80.3% | 70% | - | - |