Synaptivo

Synaps-Decision-1.0-27B-Preview

README

License: apache-2.0

Model Summary

Base Model: Qwen/Qwen3.8-27B
Base Model Revision: 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
Adapter Type: LoRA (PEFT), task_type: CAUSAL_LM
LoRA Hyperparameters: Rank $r = 32$ , $α = 64$ , dropout = $0.05$
Target Modules: Regex pattern matching self-attention (q_proj, k_proj, v_proj, o_proj), linear-attention (in_proj_qkv, in_proj_z, out_proj), and MLP (gate_proj, up_proj, down_proj)
Training Recipe: 4,000 training examples, 500 validation examples; batch size effective 32 (micro-batch 1, gradient accumulation 32); learning rate $1 \times 1 0 - 4$ with cosine decay and 100 warmup steps; best validation checkpoint at step 800 (validation loss 0.06001134521204337).
Adapter Binary: adapter_model.safetensors (870,427,776 bytes, SHA-256: cce9634bc44efaa60d9a3d0c971cb991b0b15237809f6cc8973d8315284f668c).

Evaluation Results

The following metrics reflect frozen, deterministic point evaluations comparing the unadapted base model (Qwen/Qwen3.8-27B) with Synaps-Decision-1.0-27B-Preview (ArmB checkpoint step 800).

Open Cohort Point Comparison (Descriptive)

Evaluated under frozen precision (evaluate_dataset, seed 20261022, qev_round6 prediction mode; descriptive point evidence without resampling intervals).

Table with columns: Cohort, Model, I_tiers, Calibration Score, Capability Proxy
Cohort	Model	I_tiers	Calibration Score	Capability Proxy
`l2dev` (n=438)	Base	58.6609 (`58.66090864325461`)	84.3721 (`84.37210212215678`)	71.5165 (`71.51650538270569`)
`l2dev` (n=438)	Preview	72.7350 (`72.73505225981928`)	80.9325 (`80.93255572223002`)	76.8338 (`76.83380399102465`)
`l2dev` (n=438)	Delta	+14.0741 (`+14.074143616564676`)	-3.4395 (`-3.439546399926755`)	+5.3173 (`+5.317298608318964`)
`retencja` (n=301)	Base	47.1383 (`47.138361712123576`)	81.0304 (`81.03041622974307`)	64.0843 (`64.08438897093332`)
`retencja` (n=301)	Preview	66.6104 (`66.61044562233683`)	80.4484 (`80.4484349495273`)	73.5294 (`73.52944028593205`)
`retencja` (n=301)	Delta	+19.4720 (`+19.47208391021325`)	-0.5820 (`-0.5819812802157713`)	+9.4451 (`+9.445051314998736`)

Tacit PL Evaluation (Paired Raw Results, n=602)

Evaluated on the fixed Tacit PL development split (602 items) using pinned Tacit v0.2.0 evaluation and scoring with original option mapping (no fitting, tuning, or checkpoint selection on this split; no paired statistical significance claims).

Table with columns: Model, Correct / Total, Raw Accuracy, Raw Brier Score, Raw Expected Calibration Error (ECE)
Model	Correct / Total	Raw Accuracy	Raw Brier Score	Raw Expected Calibration Error (ECE)
Unadapted Base	564 / 602	93.6877% (`0.9368770764119602`)	0.0914447 (`0.09144470347798313`)	0.0090475 (`0.009047515610592607`)
Preview (Step 800)	569 / 602	94.5183% (`0.9451827242524917`)	0.0751002 (`0.07510023204512284`)	0.0212562 (`0.02125617179612843`)
Delta (Preview - Base)	+5	+0.8306 pp (`+0.830564784053156`)	-0.0163445 (`-0.01634447143286029`, improved)	+0.0122087 (`+0.012208656185535825`, regressed)

Understanding the Metrics and Calibration Regression

Descriptive Nature: These evaluations are descriptive point comparisons obtained under frozen benchmark configurations. They do not constitute general claims of statistical superiority, nor do they reflect evaluations on sealed final holdouts.
Calibration Regression: Across both open cohorts (l2dev calibration score drops by -3.4395; retencja drops by -0.5820) and Tacit PL (raw ECE worsens from 0.0090 to 0.0213, a regression of +0.0122), probability calibration regressed after adaptation. The empirical evaluation results describe this observed difference but do not establish its cause. Post-hoc calibration procedures were not evaluated in this release, and no specific recalibration method (such as temperature scaling or Platt scaling) is prescribed.
No Competitive or Speed Claims: No claims of "#1 rank", competitive dominance, or inference speed/latency are asserted.

Training Data and Provenance

Training Distribution: The adapter was trained on 4,000 synthetic examples (with 500 validation examples) produced by procedural generation engines (deterministic rules engine, exact mathematical logic, and structured item construction).
Procedural Trace: In the active pipeline trace, synthetic items were generated procedurally without external LLM teacher calls.
Provenance Boundaries: The historical authorship of the initial underlying prompt templates and the precise historical execution runtime that originally recorded raw pilot traces are undocumented/unknown.
Overlap Audit: An open-cohort overlap audit verified 0 shared item IDs, 0 shared normalized problem states, and 0 shared serialized prompt texts between the 4,000 training / 500 validation items and the open 739 evaluation cohort. This confirms absence of exact-representation leakage within the audited open cohorts, but does not provide a universal or protected-set decontamination guarantee across unobserved benchmarks.

Architecture and Loader Compatibility

The base model Qwen/Qwen3.8-27B uses a hybrid transformer architecture consisting of 64 total layers:

48 linear-attention layers (GatedDeltaNet)
16 full self-attention layers

Important Loader Instructions

Use AutoModelForCausalLM: Always instantiate the base model via AutoModelForCausalLM.from_pretrained(...). In Transformers, this resolves cleanly to Qwen3_5ForCausalLM with layer modules structured under model.layers.<idx>.
Do Not Instantiate Qwen3_5ForConditionalGeneration Directly: Loading the base class Qwen3_5ForConditionalGeneration directly places layers under language_model.layers.<idx>, which creates an incompatible key mismatch with the adapter's trained parameter keys (base_model.model.model.layers.<idx>).
vLLM Dynamic LoRA Limitation: Dynamic runtime LoRA serving in vLLM (--enable-lora) does not support the unfused linear-attention projection targets (in_proj_qkv, in_proj_z, out_proj) present in this hybrid architecture. If deploying with vLLM or SGLang, users should first merge the adapter weights into the base model using PeftModel.merge_and_unload(safe_merge=True) and save the resulting full model weights prior to serving. No pre-merged weights are stored in this repository.

Usage Example

Model Loading (LOAD / GENERIC DEMO ONLY)

The following snippet demonstrates how to load Qwen/Qwen3.8-27B with Synaps-Decision-1.0-27B-Preview using Hugging Face Transformers and PEFT.

LOAD / GENERIC DEMO ONLY: This snippet demonstrates model loading and adapter attachment only. It is not the measured decision prompt, probability readout, or benchmark evaluation recipe, and does not reproduce decision evaluation metrics.

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_id = "Qwen/Qwen3.8-27B"base_revision = "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0"adapter_id = "Synaptivo/Synaps-Decision-1.0-27B-Preview" # Load tokenizer from pinned base model revisiontokenizer = AutoTokenizer.from_pretrained(    base_model_id,    revision=base_revision,) # Load base model in bfloat16 using AutoModelForCausalLMbase_model = AutoModelForCausalLM.from_pretrained(    base_model_id,    revision=base_revision,    torch_dtype=torch.bfloat16,    device_map="auto",) # Attach the LoRA adaptermodel = PeftModel.from_pretrained(base_model, adapter_id)model.eval()

Training Objective and Evaluation Readout Format

Training Objective: The adapter was trained via next-token negative log-likelihood (NLL) specifically on the decision option code token immediately following the assistant prefill Answer: . It was not trained on general conversational chat turns, multi-turn dialogue, or full-completion SFT objectives.
No General Chat Capability Claim: The adapter does not provide general conversational instruction following, open-domain chat, or general dialogue capabilities.
Evaluation Readout Protocol: The measured evaluation protocol formats decision items into structured chat messages:
- System prompt: "You decide one typed question from the state alone. Choose exactly one option. The reply is the option code and nothing else."
- User message: Formatted state, question, and candidate options.
- Assistant prefill: "Answer: "
- Request flags: continue_final_message=True and add_generation_prompt=False (passed via request body / extra_body).
- Readout: Probabilities are obtained from the first generated token logprobs using a conditional softmax over option codes at temperature $T = 1.0$ , rather than free-form text generation.

Runtime and Hardware Requirements

Tested Environment:
- transformers: 5.17.0
- peft: 0.21.2
- torch: 2.13.0+cu129 (Recorded from actual release-facts testing; these indicate the verified execution environment without asserting version minimum compatibility boundaries).
Precision: bfloat16
Hardware and Memory Considerations: The 27B parameter base model with attached adapter in bfloat16 requires substantial GPU VRAM (~54 GB for model weights alone, with additional headroom required for activations, KV cache, and runtime overhead during execution). Practical execution requires adequate VRAM headroom beyond the static weight footprint. No commercial hardware purchase recommendations are made.

Economics and Pricing Limits

Free submission to public evaluation queues does not constitute proof of eligibility for specific price-class benchmarks or efficiency ratings. Inference costs reflect standard 27B parameter model execution. No pricing-tier eligibility or speed/latency claims are made.

License and Attribution

Adapter License: Apache License 2.0 (see LICENSE).
Base Model License: Upstream Qwen/Qwen3.8-27B is licensed by the Qwen team under the Apache License 2.0. The upstream repository ships without a NOTICE file.
Attribution: This package includes its own NOTICE file providing attribution for the base model and adapter:
- Base model: Qwen/Qwen3.8-27B (revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0), Apache License 2.0.
- Adapter: Based on Synaps-Decision by Synaptivo sp. z o.o.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Synaptivo

Model Tree

Base

Qwen/Qwen3.8-27B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality