GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
arun11karthik
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
arun11karthik
Model Tree
Qwen/Qwen2.5-Coder-7B
Input Modalities
Output Modalities
Supported Functionality
| Model | Base | Params | Context |
|---|---|---|---|
| CellSense-FIM 0.5B | Qwen2.5-Coder-0.5B | 0.5B | 32K |
| CellSense-FIM 1.5B | Qwen2.5-Coder-1.5B | 1.5B | 32K |
| CellSense-FIM 3B | Qwen2.5-Coder-3B | 3B | 32K |
| CellSense-FIM 7B (this model) | Qwen2.5-Coder-7B | 7B | 32K |
Evaluated on the held-out test split of the CellSense FIM dataset. CellSense-FIM is compared against its base model
(Qwen2.5-Coder-7B) and a same-size general model (Qwen3-8B).
Fine-tuning on notebook-native FIM context yields large gains on the metrics that track real completion quality — edit similarity (0.07 → 0.74) and BLEU (4.9 → 57.0) — while improving token accuracy and CodeBLEU outright over the base model. At 7B, CellSense-FIM posts the strongest scores in the family.
| Model | Bits per Byte ↓ |
|---|---|
| Qwen2.5-Coder-7B (base) | 0.234 |
| Qwen3-8B (base) | 1.042 |
| CellSense-FIM 7B (ours) | 0.169 |
The model uses the Qwen2.5-Coder FIM sentinels. For a single-file completion:
markdown
<|fim_prefix|>{code before the cursor}<|fim_suffix|>{code after the cursor}<|fim_middle|>
For repository / local-import-aware completion, prepend the relevant files before the FIM block:
markdown
<|repo_name|>{repo}<|file_sep|>{path/to/helper.py}{contents of helper.py}<|file_sep|>{path/to/notebook_cell}<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>
💡 In practice you don't assemble this by hand — the CellSense JupyterLab plugin builds the repository-, import-, and task-aware context and emits exactly this format (see Serving with vLLM + CellSense below).
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "arun11karthik/cellsense-fim-7b"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")prefix = "import pandas as pd\ndf = pd.read_csv('data.csv')\n"suffix = "\ndf.head()\n"prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
The intended way to use this model is to serve it with vLLM and point the CellSense JupyterLab plugin at the server over its OpenAI-compatible API. CellSense handles all of the repository-, import-, and task-aware context assembly and the FIM prompt formatting for you.
vLLM exposes an OpenAI-compatible endpoint, which is exactly what CellSense's openai_compatible
provider expects:
bash
pip install vllmvllm serve arun11karthik/cellsense-fim-7b \--served-model-name cellsense-fim-7b \--max-model-len 32768 \--port 8000
This serves the API at http://localhost:8000/v1.
In the CellSense settings panel (left sidebar in JupyterLab), open Basic Settings and configure the OpenAI Compatible provider:
| Setting | Value |
|---|---|
| Provider | OpenAI Compatible |
| Base URL | http://localhost:8000/v1 |
| API Key | any non-empty string (vLLM ignores it, e.g. EMPTY) |
| Model | cellsense-fim-7b (must match --served-model-name) |
| Model Family | qwen2.5-coder |
Click Save & Apply, then start typing in a notebook cell — ghost-text completions from your local model appear inline. Press Tab to accept.
🌐 The same setup works for a remote vLLM server: serve the model on your GPU box, expose port
8000, and set CellSense's Base URL tohttp://<host>:8000/v1.
To confirm the endpoint works before wiring up CellSense, query it directly with the FIM prompt:
bash
curl http://localhost:8000/v1/completions \-H "Content-Type: application/json" \-d '{"model": "cellsense-fim-7b","prompt": "<|fim_prefix|>import pandas as pd\ndf = pd.read_csv(\"data.csv\")\n<|fim_suffix|>\ndf.head()\n<|fim_middle|>","max_tokens": 128,"temperature": 0.0}'
For fully local, no-GPU-required inference, GGUF builds are published at
arun11karthik/cellsense-fim-7b-GGUF.
Ollama can pull and run these directly from the Hugging Face Hub — no manual download or Modelfile
required. This is the recommended path for running CellSense entirely on your own machine: with Ollama,
no code or context ever leaves your computer.
| Quantization | Size (approx.) | Notes | Pull with |
|---|---|---|---|
Q5_K_M | 5.4 GB | Good quality / size trade-off | ollama run hf.co/arun11karthik/cellsense-fim-7b-GGUF:Q5_K_M |
Q8_0 | 8.1 GB | Near-lossless 8-bit quantization (recommended) | ollama run hf.co/arun11karthik/cellsense-fim-7b-GGUF:Q8_0 |
BF16 | 15.2 GB |
Install Ollama, then pull a quantization (this also registers the model so CellSense can use it):
bash
ollama pull hf.co/arun11karthik/cellsense-fim-7b-GGUF:Q5_K_M
By default Ollama serves its API at http://localhost:11434. The model name as it appears in
ollama list — hf.co/arun11karthik/cellsense-fim-7b-GGUF:Q5_K_M — is what you'll enter into
CellSense below.
bash
pip install jupyterlab-cellsensejupyter lab
See the CellSense repository for full installation options.
Open the CellSense panel from the left sidebar in JupyterLab, go to Basic Settings, and configure the Ollama provider:
| Setting | Value |
|---|---|
| Provider | Ollama |
| Base URL | http://localhost:11434 |
| Model Family | cellsense |
| Model | hf.co/arun11karthik/cellsense-fim-7b-GGUF:Q5_K_M (must match the tag in ollama list) |
✅ Set Model Family to
cellsense. CellSense now ships first-class support for the CellSense-FIM models, so the plugin builds prompts in exactly the repository-, import-, and task-aware FIM format these models were trained on — no extra configuration needed.
Click Save & Apply, then start typing in a notebook cell — ghost-text completions from your local model appear inline. Press Tab to accept.
To confirm Ollama is serving the model with the correct FIM format before wiring up CellSense, query it directly:
bash
curl http://localhost:11434/api/generate -d '{"model": "hf.co/arun11karthik/cellsense-fim-7b-GGUF:Q5_K_M","prompt": "<|fim_prefix|>import pandas as pd\ndf = pd.read_csv(\"data.csv\")\n<|fim_suffix|>\ndf.head()\n<|fim_middle|>","stream": false,"options": { "temperature": 0.0, "num_predict": 128 }}'
Qwen/Qwen2.5-Coder-7BCellSense-FIM was fine-tuned on a private corpus of fill-in-the-middle examples mined from Jupyter notebooks. The dataset is kept private as a precaution: PII masking was applied across the corpus and more rigororusly verified on the sampled subset used for training, but full masking across the entire source corpus has only been preliminarily checked and is not guaranteed.
As with any model trained on scraped code, this model may reproduce content from its training data, including any imperfectly masked sensitive strings. If you observe any leakage, please report it via the project's issue tracker (see the GitHub link above).
The finetuned weights are released under the GNU 3.0 License, whereas the base model was released under the Apache 2.0 license.
bibtex
@misc{cellsense-fim,title = {CellSense-FIM: Notebook-Native Fill-in-the-Middle Code Completion},author = {Arun Karthik},year = {2026},howpublished = {\url{https://huggingface.co/arun11karthik/cellsense-fim-7b}}}
| Full bfloat16 precision — highest quality |
ollama run hf.co/arun11karthik/cellsense-fim-7b-GGUF:BF16 |