zhifeixie

VoiceMem_MF_Qwen3_6_35B_A3B_Qlora

Deploy Dedicated

README

License: apache-2.0

Highlights

Memory-grounded generation — trained on multi-turn SFT data where the context is Top-K retrieved memory, not raw conversation transcripts.
Adapter-only, 180 MB — rank-32 LoRA over the language-model tower; the base model is untouched and separately licensed.
Reproducible — the training script and default hyperparameters that produced checkpoint-3318 ship in the code repo (finetune/train.py).
Drop-in with the VoiceMem stack — scripts/realtime_funasr_qwen.py runs mic → streaming memory prefetch → memory-grounded reply.

Model Overview

Type: LoRA / PEFT adapter for a Causal Language Model
Base model: Qwen/Qwen3.6-35B-A3B
Training stage: QLoRA supervised fine-tuning
Adapter
- Rank (r): 32
- Alpha: 64
- Dropout: 0.05
- Bias: none
- Task type: CAUSAL_LM
- Target modules: attention and FFN projections of the language-model tower (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, in_proj_*, , )
Training
- Checkpoint: checkpoint-3318 (2 epochs, 3,318 global steps, seed 42)
- Precision: bf16, gradient checkpointing on
- Optimizer: adamw_torch_fused, lr 2e-4, cosine schedule, warmup ratio 0.03
- Weight decay: 0.1, Adam betas (0.9, 0.95)
- Batch: 8 per device × 2 gradient accumulation steps
- Max sequence length: 2,048
Data: multi-turn SFT JSONL. Training data are not released. Provenance, licenses, and consent status are documented in the code repository before any data release.

Quickstart

bash
pip install transformers peft torch

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "Qwen/Qwen3.6-35B-A3B"
adapter_id = "zhifeixie/VoiceMem_SLM_Qwen25_omni"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
    base_model_id, trust_remote_code=True, dtype="bfloat16", device_map="auto"
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

With the VoiceMem memory system

The adapter expects retrieved memory in its context. Use VoiceMem to produce it:

bash
pip install voicemem funasr sounddevice
hf download zhifeixie/VoiceMem_Default_Models_Env --local-dir ./models

export OPENAI_API_KEY=sk-...          # write-side fact extraction only; retrieval stays local
python scripts/realtime_funasr_qwen.py

Train your own adapter

The defaults in the training script are the ones that produced checkpoint-3318; running it unchanged reproduces this adapter.

bash
pip install trl peft datasets accelerate bitsandbytes
python finetune/train.py --data data/train.jsonl

Data format, VRAM requirements, and how to swap the base model: see finetune/README.md.

Evaluation

This adapter

On AudioMC INFERENCE_MEMORY (132 conversations, 233 rubric criteria), checkpoint-3318 satisfies 97 / 233 (41.6%) criteria. Under the same protocol, GPT-4o-mini satisfies 96 / 233 (41.2%) — an improvement of 1 criterion, or +0.43 percentage points. GPT-4o-mini is the judge for both runs.

[!Note] This is a small margin on a single benchmark. Treat it as evidence that the adapter does not regress against a strong baseline, not as evidence of a large capability gain.

The VoiceMem system

The numbers below measure the memory system, not this adapter in isolation. They are reproduced here for context; see the code repository for the protocol.

Table with columns: Metric, VoiceMem, Mem0
Metric	VoiceMem	Mem0
LoCoMo accuracy (Top-5 memories)	91.2%	61.68%
PersonaMem accuracy	69.44%	—
Retrieval latency	134 ms	1,440 ms
Memory tokens per turn	302	6,956

Files

Table with columns: File, Description
File	Description
`adapter_model.safetensors`	LoRA weights (180 MB)
`adapter_config.json`	PEFT configuration with a public base-model identifier
`training_config.json`	Training and release manifest. Deliberately not named `config.json`, so no loader mistakes this adapter-only repo for a full `transformers` model.
`checksums.sha256`	SHA-256 of the two adapter files
`LICENSE`

Optimizer shards, DeepSpeed state, RNG state, scheduler state, and training logs are not model artifacts and are not published here.

Verify a download:

bash
sha256sum -c checksums.sha256

Intended Use and Limitations

Intended use. Research on long-term conversational memory and memory-grounded answer generation.

Limitations.

This is a research adapter. It has not been evaluated for safety-critical use and does not replace the base model's own safety documentation.
It may reproduce biases and errors from both the training data and the base model.
It is tuned for memory-grounded replies. Outside that setting it has no evaluated advantage over the base model.
The adapter cannot verify the memories it is given. Wrong or stale retrievals produce confident wrong answers.
Evaluation covers Chinese and English only.

License

The adapter is released under the Apache License 2.0. The base model carries its own license, which you must accept separately — this release grants no rights to it.

Citation

If you find this adapter useful, please cite the VoiceMem repository:

bibtex
@software{voicemem,
    title  = {VoiceMem: Streaming Dual-Brain Memory for Real-Time Voice Agents},
    author = {{VoiceMem Team}},
    year   = {2026},
    url    = {https://github.com/lang-jiaqi/Voicemem_open}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

zhifeixie

Model Tree

Base

Qwen/Qwen3.6-35B-A3B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality