1. Overview
Baphomet-1.0-4B is a model produced by linearly combining 8 task-vectors (T = W_donor − W_base) into the base model Qwen/Qwen3.5-4B.
According to a 2025 paper (hitit et al., 2025) on model merging, pure Task Arithmetic is the only method that consistently improves performance across "in-the-wild" models. Based on that paper, I merged high-quality community fine-tuned versions of Qwen3.5-4B.
W_merged = W_base + Σ αᵢ · Tᵢ (αᵢ = 0.1375)
The configured weights sum to 0.1375 × 8 = 1.10. According to goatmerge.json, normalize: true, rescale: true are enabled, so normalized coefficients are applied. Refer to the goatmerge implementation for the exact computation.
No claim is made about improved performance or inheritance of specific capabilities. No quantitative evaluation has been performed.
2. Merge configuration
2.1 Base model and settings
Table with columns: Item, Value| Item | Value |
|---|
| Base | Qwen/Qwen3.5-4B |
| Base fingerprint | 62de7a218d003afe6dffca82532c20150488b68ec629e72f9b50a0615dd7c8c5 |
| Tool | goatmerge (format v1.1.0) |
| Method | linear, density=1.0, normalize=true, rescale=true |
| Chunk | 8388608 elements, drop_caches=true |
2.2 Task-vector list (α = 0.1375 each)
The table below transcribes the extraction source (source_model) recorded in each TV's goatmerge.json. No statement of effect is made.
Table with columns: #, Task-vector, Recorded source_model| # | Task-vector | Recorded source_model |
|---|
| 1 | Agents-A1-4B-TV | InternScience/Agents-A1-4B |
| 2 | DeepSeek-V4-Pro-Qwen3.5-4B-TV | Jackrong/DeepSeek-V4-Pro-Qwen3.5-4B |
| 3 | Qwen3.8-4B-Distill-TV | empero-ai/Qwen3.8-4B-Distill |
| 4 |
base: Qwen/Qwen3.5-4Bout: ./Baphomet-1.0-4B-k8 merge_method: linearchunk_elements: 8388608drop_caches: trueskip_fingerprint_check: false tv: - dir: ./Task-Vectors/Agents-A1-4B-TV weight: 0.1375 - dir: ./Task-Vectors/DeepSeek-V4-Pro-Qwen3.5-4B-TV weight: 0.1375 - dir: ./Task-Vectors/Qwen3.8-4B-Distill-TV weight: 0.1375 - dir: ./Task-Vectors/Qwopus3.5-4B-Coder-TV weight: 0.1375 - dir: ./Task-Vectors/Qwopus3.5-4B-v3-TV weight: 0.1375 - dir: ./Task-Vectors/AREX-Turbo-TV weight: 0.1375 - dir: ./Task-Vectors/NeoHorse-1-4B-TV weight: 0.1375 - dir: ./Task-Vectors/tmax-4B-TV weight: 0.1375
Note: NeoHorse-1-4B uses a text-only key layout, so its keys were converted from model.* to model.language_model.* with align_neohorse.py before extraction. See Section 5 of the parent directory README for details.
3. Model specifications
Excerpt from config.json.
Table with columns: Item, Value| Item | Value |
|---|
| Architecture | Qwen3_5ForConditionalGeneration |
| Text layers | 32 layers, hidden size 2560, heads 16Q / 4KV (GQA), head_dim 256 |
| Attention layer types | Repeating pattern of 3 linear_attention layers followed by 1 full_attention layer |
| FFN | intermediate 9216, SiLU, |
4. Included files
Table with columns: File, Size, Description| File | Size | Description |
|---|
model-00001-of-00010.safetensors through model-00010… | Approx. 8.9GB total | Model weights (bf16, 10 shards) |
model.safetensors.index.json | 66KB | Shard index |
Baphomet-1.0-4B-q8_0.gguf | 4.2GB | Q8_0 quantization for llama.cpp |
config.json | 3.1KB |
5. Usage
from transformers import AutoModelForVision2Seq, AutoProcessor model_id = "Baphomet-1.0-4B" # Replace with a local path or repository name model = AutoModelForVision2Seq.from_pretrained( model_id, dtype="auto", device_map="auto", trust_remote_code=True,)processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True) messages = [ {"role": "user", "content": [ {"type": "image", "image": "https://example.com/cat.png"}, {"type": "text", "text": "Describe this image in English."}, ]},]inputs = processor.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=512)print(processor.decode(out[0], skip_special_tokens=False))
5.2 llama.cpp (GGUF)
# Interactive chatllama-cli -m Baphomet-1.0-4B-q8_0.gguf -cnv -c 8192 # Server modellama-server -m Baphomet-1.0-4B-q8_0.gguf -c 32768 --port 8080
5.3 vLLM
vllm serve ./Baphomet-1.0-4B-k8 --max-model-len 32768 --trust-remote-code
<|im_start|>systemYou are a helpful assistant.<|im_end|><|im_start|>userWrite a parallel downloader in Rust<|im_end|><|im_start|>assistant<think>Organizing the requirements…</think>```rust// code here```<|im_end|>
Tool calls may be emitted in <tool_call><function=…> form and reasoning in <think>…</think>. Refer to chat_template.jinja for the exact format.
6. Benchmark
Table with columns: Suite, Condition, Base (Qwen/Qwen3.5-4B), Baphomet-1.0-4B, Notes| Suite | Condition | Base (Qwen/Qwen3.5-4B) | Baphomet-1.0-4B | Notes |
|---|
| TBD | Same prompts and settings as base | TBD | TBD | To be recorded with harness version, seed, and date |
| TBD | Same prompts and settings as base | TBD | TBD | To be recorded with harness version, seed, and date |
Results, when available, will be recorded here with the evaluation date, harness and dataset versions, sampling parameters, and number of runs. The base model will be evaluated under identical conditions for comparison.
7. Limitations
- The model is the result of a linear merge; there is no guarantee that characteristics of individual donor models are preserved.
- It may inherit the license terms and biases of each donor and the base model. Determine the scope of use after reviewing the license of each model.
- Benchmark evaluation is in progress (see Section 6). No results are reported yet, and no performance claims are made.
- Use for final decisions in high-risk domains such as medical or legal matters is not intended.
8. Credits
Table with columns: Role, Model| Role | Model |
|---|
| Base | Qwen/Qwen3.5-4B |
| Donors | InternScience/Agents-A1-4B, Jackrong/DeepSeek-V4-Pro-Qwen3.5-4B, empero-ai/Qwen3.8-4B-Distill, Jackrong/Qwopus3.5-4B-Coder, Jackrong/Qwopus3.5-4B-v3, BAAI/AREX-Turbo, TokenRhythm/NeoHorse-1-4B, allenai/tmax-4b |
| Merge tool | GoatMerge |
| References | O. K. Hitit, C. Sankar, Y. K. Kim, A. Risteski, and A. Elgohary, "A systematic study of in-the-wild model merging for large language models," arXiv preprint arXiv:2511.21437, 2025. |