GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
chartreuse-verte
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
chartreuse-verte
Model Tree
ibm-granite/granite-4.0-350m-base
Input Modalities
Output Modalities
Supported Functionality
| Path | Format | Use with |
|---|---|---|
/ | safetensors bf16, plain granite arch | transformers |
GGUF/orb-human-typeahead-350m-v1.2-Q8_0.gguf | GGUF Q8_0 (default) | llama.cpp / llama-cpp-python |
The original fine-tune used the granitemoehybrid architecture; since this
variant is attention-only and dense, the weights are republished here as the
equivalent plain granite architecture (logit-identical, verified), which
loads everywhere without extras.
Plain text, no chat template. Optional character summary, a marker line, name-prefixed turns, and finally the user's draft — the model continues the draft. Cut the suggestion at the first newline.
markdown
<character summary, optional>***Roleplay chat below***Sylvara: *She looks down from the watchtower and sees you.*Traveler: *I approach the encampment.*Sylvara: *She lowers her bow as you approach the gate.* "State your business, traveler."Traveler: *I raise both hands slowly and
A completion looks like step into the torchlight, keeping my voice low.*
Notes:
Greedy, short budget, stop at newline — mirrors how it was trained and evaluated:
python
from llama_cpp import Llamallm = Llama("GGUF/orb-human-typeahead-350m-v1.2-Q8_0.gguf", n_ctx=1024)out = llm.create_completion(prompt, max_tokens=12, stop=["\n"], temperature=0.0)print(out["choices"][0]["text"])
Or with transformers:
python
from transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("chartreuse-verte/orb-human-typeahead-350m-v1.2")model = AutoModelForCausalLM.from_pretrained("chartreuse-verte/orb-human-typeahead-350m-v1.2")ids = tok(prompt, return_tensors="pt").input_idsout = model.generate(ids, max_new_tokens=12, do_sample=False)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).split("\n")[0])
Scored on a held-out validation set of roleplay conversations (120 prompts, conversation-disjoint from training; greedy, 12-token budget, suggestions cut at newline). word-EM@k = fraction of prompts where the first k words of the suggestion exactly match what the user really typed next; prefix-chars = mean length of the exactly-matching leading characters.
| Metric | orb-human-typeahead-350m-v1.1 | this model |
|---|---|---|
| word-EM@1 | 0.333 | 0.358 |
| word-EM@2 | 0.133 | 0.150 |
| word-EM@3 | 0.075 | 0.075 |
| prefix-chars | 3.92 | 4.12 |
| completion perplexity | 8.83 | 9.00 |
v1.2 is a data-only refresh of v1.1 (same 350M granite base) whose purpose was to flatten the completion distribution further — to stop the model defaulting to a small set of high-frequency phrasings. The validation set is itself drawn from the generator-styled distribution, so its gold completions over-represent exactly those defaults; a flatter model is expected to pay a small price here. v1.2 holds or gains on every exact-match column anyway (word-EM@1 0.333 → 0.358), with perplexity essentially unchanged (8.83 → 9.00).
Read the table as a no-regression check on fluency and next-word accuracy, not as a quality ranking — exact agreement with one recorded continuation cannot score suggestion variety, and creative roleplay is high-entropy enough that many suggestions the metric counts as misses are perfectly good ones.
Two-stage full fine-tune of granite-4.0-350m-base: stage 1 on a mostly-synthetic roleplay corpus (self-chat generated to cover the on-scene, in-voice register real users type in), then stage 2 on a smaller private in-domain set in the serve-time prompt format (with stage-1 replay to limit forgetting). Over v1.1, stage 1's completion distribution was flattened further: a 3-gram repetition cap retires canned mid-phrase filler that the earlier first-word cap could not see, post-dialogue beats got real supervision instead of being capped into starvation, and generator punctuation tells (em-dashes, curly quotes) plus non-English leakage are normalized out rather than dropped. Examples are user turns split at word boundaries; loss is on the continuation. Suggestions are single-line by construction (completions end at newline).