GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AvinashRicky
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
AvinashRicky
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Parameter | Specification |
|---|---|
| Model Size | 183,926,400 parameters (183M) |
| Architecture | Autoregressive Decoder-only Transformer (GPT2LMHeadModel compatible) |
| Layer Count | 16 Transformer Layers |
| Hidden Dimension (dmodel) | 896 |
| Attention Heads | 14 heads (head dimension = 64) |
| Context Window | 1,024 tokens |
| Vocabulary | 32,009 custom BPE tokens (including 9 hardware control tokens) |
| Pretraining Volume | ~5.0 Billion tokens (English Wikipedia + FineWeb-Edu subset) |
| Alignment Dataset | 25,850 multi-step hardware trajectories |
| Storage Engine | Local SQLite 3 FTS5 (WAL mode, normal synchronous disk writes) |
| SSD Latency | 1.18 milliseconds (NVMe average read latency) |
| RAM Footprint | ~0.4 GB (runs comfortably on CPU or edge devices) |
AViGPT manages external execution through nine dedicated vocabulary tokens:
text
User Instruction │ ▼[ AViGPT Neural Core (183M) ] │ ├── Emits <|intent_start|> ... <|intent_end|> (Goal framing) ├── Emits <|mem_query|> ... <|mem_query_end|> │ │ │ ▼ │ [ NVMe SSD / SQLite FTS5 Engine ] ── Latency: 1.18 ms │ │ ├── Emits <|mem_payload|> ... <|mem_payload_end|> (Injects factual record) ├── Emits <|calc|> ... <|calc_end|> (Optional arithmetic sandbox) │ ▼ <|synthesize|> (Produces final verified answer)| Special Token | Role | Runtime Action |
|---|---|---|
| `< | intent_start | >/< |
| `< | mem_query | >/< |
| `< | mem_payload | >/< |
| `< | calc | >/< |
| `< | synthesize | >` |
Cross-entropy loss declined from 3.3698 to 0.6935 over 1,800 steps on an NVIDIA T4 GPU:
Direct NVMe storage retrieval operates in 1.18 milliseconds, compared to 500 to 1,500 milliseconds for network-based RAG architectures:
AViGPT operates with a 0.4 GB memory footprint, running entirely on consumer CPUs without requiring dedicated GPU accelerators:
| System | Parameters | Minimum Hardware | Retrieval Latency | Knowledge Updates |
|---|---|---|---|---|
| AViGPT | 183M | 0.4 GB RAM (Any CPU) | 1.18 ms (NVMe SSD) | Immediate (0-cost disk write) |
| SmolLM-135M | 135M | 0.3 GB RAM | None (Parametric only) | Retraining required |
| LLaMA-3-8B | 8.0B | 16 GB VRAM | N/A | Retraining required |
| Standard RAG | 8B+ | 16 GB + Vector DB | 450 ms – 1,200 ms | Index re-embedding |
You can load and query the model directly via the transformers library:
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "AvinashRicky/AViGPT" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32) prompt = ( "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n" "### Instruction:\nWho is your owner and creator?\n\n### Response:\n") inputs = tokenizer(prompt, return_tensors="pt")with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=120, temperature=0.2, pad_token_id=tokenizer.eos_token_id ) print(tokenizer.decode(outputs[0], skip_special_tokens=False))To run AViGPT with active sub-millisecond SSD queries and arithmetic execution:
bash
git clone https://github.com/Avinashricky211/AviGPT.gitcd AviGPTpip install -r requirements.txtstreamlit run app.pyOr programmatically in Python:
python
import torchfrom transformers import GPT2LMHeadModel, GPT2TokenizerFastfrom autonomous_bus import AutonomousHardwareBus model_id = "AvinashRicky/AViGPT"tokenizer = GPT2TokenizerFast.from_pretrained(model_id)model = GPT2LMHeadModel.from_pretrained(model_id).to("cuda" if torch.cuda.is_available() else "cpu") # Initialize hardware bus controller with local SQLite FTS5 enginebus = AutonomousHardwareBus(model=model, tokenizer=tokenizer) # Execute query with hardware-accelerated memory retrievalresponse, metrics = bus.generate_autonomous_response("When did Apollo 11 land on the Moon?") print("Response:\n", response)print(f"SSD Retrieval Latency: {metrics['ssd_latency_ms']:.2f} ms")print(f"Total Response Latency: {metrics['total_latency_s']:.2f} s")Phase 1: Pretraining from Scratch
Phase 2: Hardware Bus Alignment
bibtex
@article{Avinash2026avigpt, title={AViGPT: Decoupling Neural Reasoning from Parametric Memory via a Sub-Millisecond Native NVMe Hardware Bus}, author={Avinash Ricky Yadlapalli}, year={2026}, journal={Zenodo}, doi={10.5281/zenodo.22856047}, howpublished={\url{https://doi.org/10.5281/zenodo.22856047}}, url={https://github.com/Avinashricky211/AviGPT}}User Instruction │ ▼[ AViGPT Neural Core (183M) ] │ ├── Emits <|intent_start|> ... <|intent_end|> (Goal framing) ├── Emits <|mem_query|> ... <|mem_query_end|> │ │ │ ▼ │ [ NVMe SSD / SQLite FTS5 Engine ] ── Latency: 1.18 ms │ │ ├── Emits <|mem_payload|> ... <|mem_payload_end|> (Injects factual record) ├── Emits <|calc|> ... <|calc_end|> (Optional arithmetic sandbox) │ ▼ <|synthesize|> (Produces final verified answer)import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "AvinashRicky/AViGPT" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32) prompt = ( "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n" "### Instruction:\nWho is your owner and creator?\n\n### Response:\n") inputs = tokenizer(prompt, return_tensors="pt")with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=120, temperature=0.2, pad_token_id=tokenizer.eos_token_id ) print(tokenizer.decode(outputs[0], skip_special_tokens=False))git clone https://github.com/Avinashricky211/AviGPT.gitcd AviGPTpip install -r requirements.txtstreamlit run app.pyimport torchfrom transformers import GPT2LMHeadModel, GPT2TokenizerFastfrom autonomous_bus import AutonomousHardwareBus model_id = "AvinashRicky/AViGPT"tokenizer = GPT2TokenizerFast.from_pretrained(model_id)model = GPT2LMHeadModel.from_pretrained(model_id).to("cuda" if torch.cuda.is_available() else "cpu") # Initialize hardware bus controller with local SQLite FTS5 enginebus = AutonomousHardwareBus(model=model, tokenizer=tokenizer) # Execute query with hardware-accelerated memory retrievalresponse, metrics = bus.generate_autonomous_response("When did Apollo 11 land on the Moon?") print("Response:\n", response)print(f"SSD Retrieval Latency: {metrics['ssd_latency_ms']:.2f} ms")print(f"Total Response Latency: {metrics['total_latency_s']:.2f} s")@article{Avinash2026avigpt, title={AViGPT: Decoupling Neural Reasoning from Parametric Memory via a Sub-Millisecond Native NVMe Hardware Bus}, author={Avinash Ricky Yadlapalli}, year={2026}, journal={Zenodo}, doi={10.5281/zenodo.22856047}, howpublished={\url{https://doi.org/10.5281/zenodo.22856047}}, url={https://github.com/Avinashricky211/AviGPT}}