What Makes KORMo-Web-11B Different?
- Built on Our Own Foundation Model: Developed from the KORMo-10B language model, trained from scratch by KAIST MLP Lab, and extended through KORMo-VL into a web agent.
- Three Capabilities in One Model: Designed to retain general LLM reasoning and VLM understanding while adding GUI interaction capabilities, rather than optimizing exclusively for web navigation.
- 10M+ Atomic-Level Web Pretraining Samples: Large-scale Korean and English web understanding training, covering individual UI elements, their locations, layouts, functions, and states.
- 100K+ Web Interaction Trajectories: Bilingual browser demonstrations for learning real-world, multi-step web tasks.
- Open and Commercially Usable: Model weights and supporting resources are openly released under Apache-2.0, allowing commercial use, modification, and redistribution subject to the license terms.
From a Korean LLM to a General-Purpose Web Agent
KORMo-Web-11B extends our independently developed Korean–English foundation model through three stages:
Table with columns: Stage, Model, Capabilities| Stage | Model | Capabilities |
|---|
| 1. Language | KORMo-10B | Korean–English LLM trained from scratch with a Korean-optimized tokenizer |
| 2. Vision | KORMo-VL | Multimodal understanding through a SigLIP vision encoder and KORMo language backbone |
| 3. Web Agent | KORMo-Web-11B | Large-scale web pretraining and action learning for visual browser interaction |
The goal is not just a better web agent, but a versatile model that can understand language, interpret visual information, and interact with the web.
We evaluated KORMo-Web-11B on 612 live-website tasks (300 English, 312 Korean) from Online-Mind2Web, using the same LLM judge across all evaluated models.
Table with columns: Model, Size, Online-Mind2Web EN, Online-Mind2Web KO| Model | Size | Online-Mind2Web EN | Online-Mind2Web KO |
|---|
| Qwen3.5-9B (base) | 9B | 24.7 | 16.7 |
| Fara-1.5-4B | 4B | 20.7 | 7.7 |
| UI-TARS-1.5-7B | 7B | 22.3 | 20.2 |
| OpenWebRL-4B | 4B | 32.7 | 15.4 |
| KORMo-Web-11B (ours) | 11B | 31.3 | 33.7 |
| GPT-6-luna (proprietary) | – | 63.0 | 65.7 |
Success rates are calculated over the full task set, including failed or incomplete runs. Each agent uses its own prompt and action format.
KORMo-Web-11B achieves 33.7% Korean task success, exceeding the strongest non-KORMo open-weight baseline listed above by 13.5 percentage points.
While our specialized KORMo-Web-4B and 9B models achieve higher web task success, KORMo-Web-11B prioritizes versatility across language, vision, and GUI interaction, rather than web-only optimization.
Why KORMo-Web-11B?
- Korean from the Ground Up: Built upon our independently pretrained Korean–English foundation model.
- General-Purpose Multimodal AI: Supports text-based language tasks, visual understanding, and browser interactions within one model.
- Visual-First Web Interaction: Operates primarily through screenshots without requiring full DOM or accessibility trees.
- Think, Then Act: Generates an inspectable reasoning step before executing browser actions.
- Commercial Use Permitted: Released under Apache-2.0, supporting academic research, commercial applications, fine-tuning, and redistribution.
- Open and Ready to Run: Includes model weights, Transformers 5 support, a vLLM inference path, browser harness, and reference inference settings.
KORMo-Web-11B — From a Korean Foundation Model to a General-Purpose Web Agent.
How to Start: Chrome Extension
The fastest way to try KORMo-Web is the KORMo-Web Chrome extension: type a request in Chrome's side panel and watch the agent work in your own browser. This video walks through the whole setup, from serving the model to running a task and controlling it from Discord.
Setup guide (about 15 min, Korean narration). Parts that showed account tokens have been blurred.
1. Serve the model on a Mac or on an NVIDIA GPU.
Mac (Apple silicon)
KORMo-Web-11B is served with vLLM on an NVIDIA GPU (see below); a Mac setup for 11B has not been tested. To try KORMo-Web on a Mac, run KORMo-Web-4B or KORMo-Web-9B with vllm-mlx 0.6.0 or later. Pick the one that fits your Mac's memory:
Table with columns: Model, Weights, Memory in use, Recommended Mac memory| Model | Weights | Memory in use | Recommended Mac memory |
|---|
KORMo-Web-4B-mlx-4bit (4-bit) | 2.8 GB | about 3.8 GB | 16 GB or more |
KORMo-Web-4B (bf16) | 9.6 GB | about 9.5 GB | 24 GB or more |
KORMo-Web-9B (bf16) | 17.5 GB | about 19 GB | 48 GB or more (32 GB is tight) |
uv venv --python 3.12 && source .venv/bin/activateuv pip install "vllm-mlx>=0.6.0" huggingface_hubMODEL=KORMo-Web-4B-mlx-4bit # or KORMo-Web-4B / KORMo-Web-9Bhf download KORMo-VL/$MODEL --local-dir ~/models/$MODELvllm-mlx serve ~/models/$MODEL --port 8010 --served-model-name KORMo-VL/$MODEL \ --enable-auto-tool-choice --tool-call-parser auto
Keep this terminal open. --served-model-name is the name the extension must request.
NVIDIA GPU (local PC or server)
vLLM needs a one-time conversion for the KORMo architecture: follow steps 1 and 2 of Run the agent (vLLM) in the Quickstart below, which serve the model as KORMo-Web-11B on port 8000.
Check that it is up with curl http://localhost:8010/v1/models (Mac) or curl http://<server>:8000/v1/models (vLLM). If you changed the port, use that port.
2. Install the extension. git clone https://github.com/MLP-Lab/KORMo-Web-Extension.git, open chrome://extensions, turn on Developer mode, click Load unpacked and choose the cloned folder. Open the GUI 에이전트 side panel, type a task and press 실행 (Run).
3. Point the extension at your model. Open the panel's 개발자 설정 (Developer settings).
- Mac: keep the server URL
http://localhost:8010/v1 and tool_choice auto. Set the model name to your --served-model-name (the default KORMo-VL/KORMo-Web-4B-mlx-4bit already matches the 4-bit build).
- NVIDIA GPU: set the server URL to
http://<server>:8000/v1 (or localhost), the model name to KORMo-Web-11B, and tool_choice to none.
4. Discord (optional). Run node relay/discord-relay.mjs in the cloned folder and follow the prompts to send tasks to the agent by mentioning a bot in a channel.
Full step-by-step instructions (Korean) are in the extension README.
Quickstart
Tested with transformers 5.12.1 and vLLM 0.23.0. The model runs on a single GPU.
transformers does not ship the KORMo text model, so this repo includes it (_modeling_kormo.py) together with a small loader, kormo_load.py:
kormo_load.register(path) registers the KORMo text model. Call it before loading the processor or the model.
kormo_load.patch_vision(model) feeds the SigLIP features after post-layernorm to the projector, as in training. Without it the model still runs but misreads screenshots.
pip install -U "transformers>=5.12" torch pillow huggingface_hubhf download KORMo-VL/KORMo-Web-11B --local-dir KORMo-Web-11B
import json, sys, torchfrom PIL import Imagefrom transformers import AutoModelForImageTextToText, AutoProcessor M = "KORMo-Web-11B"sys.path.insert(0, M)import kormo_loadkormo_load.register(M)processor = AutoProcessor.from_pretrained(M)model = AutoModelForImageTextToText.from_pretrained(M, dtype=torch.bfloat16).to("cuda").eval()kormo_load.patch_vision(model) SYSTEM = open(f"{M}/system_prompt.md").read().strip()TOOLS = json.load(open(f"{M}/tools.json"))obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n" "- Tab 0 (active): https://www.aladin.co.kr/\n\nurl: https://www.aladin.co.kr/\nscreenshot:\n")messages = [ {"role": "system", "content": SYSTEM}, {"role": "user", "content": [ {"type": "text", "text": "<user_request>\n알라딘 사이트에서 '해커스 토익 RC'를 검색한 뒤 가장 먼저 표시되는 도서의 출간일을 알려줘.\n</user_request>\n" + obs}, {"type": "image", "image": Image.open("screen.png").convert("RGB")}, {"type": "text", "text": "\n</observation>"}, ]},]inputs = processor.apply_chat_template(messages, tools=TOOLS, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=16384, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))
A real first step from our Online-Mind2Web run (Aladin home page):
<think>I see the Aladin homepage, with the search field labeled “통합검색” near the top. I have not yet searched for the book, so I still need to enter “해커스 토익 RC,” inspect the first result’s publication date, and report that date. My plan:1. Click the search field2. Enter “해커스 토익 RC” and submit the search3. Read the publication date of the first listed bookThis click targets the visible search field and is a safe, relevant first action.</think> <tool_call><function=click><parameter=x>482</parameter><parameter=y>212</parameter></function></tool_call>
Coordinates are normalized to 0–1000. Convert them to pixels with x / 1000 × width and y / 1000 × height.
Run the agent (vLLM)
vLLM does not include the KORMo architecture either. The KORMo text layers have the Llama structure, so vllm/convert_to_vllm.py renames two parameter names and writes a folder vLLM loads natively. vllm/sitecustomize.py turns SigLIP post-layernorm back on inside vLLM.
1. Convert (once)
pip install -U "vllm>=0.23.0" openai playwright safetensorsplaywright install chromiumpython KORMo-Web-11B/vllm/convert_to_vllm.py KORMo-Web-11B KORMo-Web-11B-vllm
2. Start the model server
PYTHONPATH=$PWD/KORMo-Web-11B/vllm vllm serve ./KORMo-Web-11B-vllm --served-model-name KORMo-Web-11B \ --max-model-len 65536 --limit-mm-per-prompt '{"image":1,"video":0}'
Check that the server log prints [kormo-vllm] SigLIP post_layernorm enabled.
3. Run a browser task
python KORMo-Web-11B/examples/agent_playwright.py --model KORMo-Web-11B \ --start-url https://www.danawa.com/ \ --task "다나와에서 50만 원 이하, 10인치 이상 태블릿 3개를 찾아서 가격을 알려줘"
The agent drives a headless Chromium, prints each action as it goes, and finishes with the model's answer. Add --headful to watch the browser window. The script sends tool_choice="none", which vLLM needs: the tool schema stays in the prompt and the call comes back as text (sending tools without tool_choice returns HTTP 400 unless the server was started with --enable-auto-tool-choice).
How It Works
user <user_request> task </user_request> + observation (tabs, url, screenshot)assistant <think> what I see · what is done · plan </think> <tool_call> one action </tool_call>tool action result + new observation (newest screenshot only)… repeat until finished(answer)
- Action space: 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control,
finished, and others (tools.json).
- Memory: the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
- Recommended settings: temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in
generation_config.json.
The exact message format, parsing rules and feedback strings are in HARNESS.md. Follow it if you build your own harness.
Evaluation Details
Table | |
|---|
| Tasks | Online-Mind2Web: 300 English, 312 Korean |
| Judge | GPT-6 |
Files
Table with columns: File, Description| File | Description |
|---|
model.safetensors · config.json | Weights and config (LLaVA-OneVision: SigLIP + KORMo-10B text model) |
_configuration_kormo.py · _modeling_kormo.py | KORMo text model code for transformers 5 |
kormo_load.py | register() and patch_vision() |
chat_template.jinja | Chat template (XML tool calls, thinking on) |
system_prompt.md · tools.json | System prompt and 22-tool schema |
HARNESS.md | Full harness specification |
examples/agent_playwright.py | Reference browser agent (Playwright + OpenAI-compatible API). --tool-choice none for vLLM (default); --headful shows the browser. It does not execute upload. |
vllm/convert_to_vllm.py · vllm/sitecustomize.py | Convert for vLLM and the post-layernorm fix |
generation_config.json · *processor_config.json · tokenizer*.json · special_tokens_map.json | Sampling, processor and tokenizer configs |
Acknowledgements
Built on KORMo-VL and KORMo-10B by KAIST MLP Lab. Evaluated on Online-Mind2Web and its Korean counterpart.
- KyungTae Lim, Professor at KAIST.
ktlim@kaist.ac.kr
Contributor