GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
VishwamAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
VishwamAI
Model Tree
unsloth/gemma-4-E4B-it
Input Modalities
Output Modalities
Supported Functionality
VishwamAI is a 4B-class assistant designed to run locally on modest hardware while handling four kinds of work:
Design principle: reason with the model, verify with a tool. Facts (CVEs, ATT&CK IDs, constants) should be served from a local RAG index rather than trusted from the weights.
| Base model | unsloth/gemma-4-E4B-it (Apache 2.0) |
| Method | QLoRA SFT (Unsloth + TRL), rank 16, merged to fp16 |
| Context | up to 128K (base); trained at 2048 |
| Precision | fp16 safetensors (sharded) |
| Language | English |
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "VishwamAI/vishwamai-merged"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto") msgs = [{"role": "user", "content": "Explain MITRE ATT&CK T1003.001 and give two Sysmon detections."}]inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(input_ids=inputs, max_new_tokens=300, temperature=0.7)print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))Convert the fp16 weights to a small quantized GGUF on a machine with ≥16 GB RAM:
bash
huggingface-cli download VishwamAI/vishwamai-merged --local-dir ./vishwamai-mergedpython convert_hf_to_gguf.py ./vishwamai-merged \ --outfile vishwamai-f16.gguf --outtype f16 --use-temp-file./llama-quantize vishwamai-f16.gguf vishwamai-Q4_K_M.gguf Q4_K_M# serve an OpenAI-compatible API on localhost./llama-server -m vishwamai-Q4_K_M.gguf -ngl 99 -c 8192 --host 127.0.0.1 --port 8080Then call it:
python
import requestsr = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={ "model": "vishwamai", "temperature": 0.7, "messages": [{"role": "user", "content": "Give two Sysmon detections for LSASS dumping."}]})print(r.json()["choices"][0]["message"]["content"])text
You are VishwamAI, a defensive-first security research assistant and amath/science solver for an authorized practitioner. Be precise, state uncertainty,never fabricate CVE IDs or physical constants, and use the calculator/CAS tool forany non-trivial computation.A mixed instruction set (~security ~40–50%, math/science ~20–25%, general ~15–20%, plus ~3–5% safety refusals and ~5–10% contrastive "allowed defensive" examples to reduce over-refusal). Public sources used may include Trend Micro Primus-Instruct (ODC-By), MITRE ATT&CK, and NVD/CVE-derived reasoning tasks, alongside the author's own notes. Benchmark items were held out of training.
Fine-tuning can erode alignment, so safety is treated as a system: curated refusals + contrastive allowed examples in training, an optional runtime guard model (e.g. Llama Guard / Qwen3Guard) on outputs, prompt-injection handling that treats all retrieved/tool content as untrusted data, and sandboxed tool execution (no host shell). Re-run a safety eval after every retrain/re-quantization.
Released under Apache 2.0, inheriting the license of the Gemma 4 base. Review the licenses/terms of any datasets you add before redistribution.
Provided for authorized security research, education, and lawful use only. The authors accept no liability for misuse. You are responsible for complying with all applicable laws and for having authorization to test any system.
from transformers import AutoModelForCausalLM, AutoTokenizerimport torch model_id = "VishwamAI/vishwamai-merged"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto") msgs = [{"role": "user", "content": "Explain MITRE ATT&CK T1003.001 and give two Sysmon detections."}]inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(input_ids=inputs, max_new_tokens=300, temperature=0.7)print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))huggingface-cli download VishwamAI/vishwamai-merged --local-dir ./vishwamai-mergedpython convert_hf_to_gguf.py ./vishwamai-merged \ --outfile vishwamai-f16.gguf --outtype f16 --use-temp-file./llama-quantize vishwamai-f16.gguf vishwamai-Q4_K_M.gguf Q4_K_M# serve an OpenAI-compatible API on localhost./llama-server -m vishwamai-Q4_K_M.gguf -ngl 99 -c 8192 --host 127.0.0.1 --port 8080import requestsr = requests.post("http://127.0.0.1:8080/v1/chat/completions", json={ "model": "vishwamai", "temperature": 0.7, "messages": [{"role": "user", "content": "Give two Sysmon detections for LSASS dumping."}]})print(r.json()["choices"][0]["message"]["content"])You are VishwamAI, a defensive-first security research assistant and amath/science solver for an authorized practitioner. Be precise, state uncertainty,never fabricate CVE IDs or physical constants, and use the calculator/CAS tool forany non-trivial computation.