GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
buddhist-nlp
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
buddhist-nlp
Model Tree
Input Modalities
Output Modalities
Supported Functionality
The model expects asymmetric inputs:
Query (instruction + query):
<instruct>Please find the semantically most similar text in {language}.\n<query>{sentence_text}
where {language} is a full language name (e.g. "Sanskrit", "Tibetan", "English", "Chinese", "Pali", "Hindi") and {sentence_text} is the query string.
Corpus:
Use the raw sentence (or passage) text only, with no <instruct> or <query> wrapper.
python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigimport torchmodel_path = "gemma2-mitra-embedding"quantization_config = BitsAndBytesConfig(load_in_8bit=True,llm_int8_threshold=6.0,llm_int8_has_fp16_weight=False,)model = AutoModelForCausalLM.from_pretrained(model_path,quantization_config=quantization_config,device_map={"": 0},torch_dtype=torch.float16,trust_remote_code=True,)tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)if tokenizer.pad_token is None:tokenizer.pad_token = tokenizer.eos_token# Query encoding (one sentence)language = "Sanskrit" # or Tibetan, English, Chinese, Pali, Hinditext = "Your query sentence here."prompt = f"<instruct>Please find the semantically most similar text in {language}.\\n<query>{text}"inputs = tokenizer(prompt, return_tensors="pt", padding=True, truncation=True, max_length=512).to(model.device)with torch.no_grad():outputs = model(**inputs, output_hidden_states=True)# Last token embedding (last non-padded)last_token_idx = inputs["attention_mask"].sum(dim=1) - 1embedding = outputs.hidden_states[-1][torch.arange(last_token_idx.size(0)), last_token_idx]# L2-normalize for cosine similarityembedding = embedding / embedding.norm(dim=-1, keepdim=True)
For corpus sentences, pass only the raw text (no <instruct>/<query>), then take the last token hidden state and L2-normalize the same way.
Alternatively, use FlagEmbedding’s FlagLLMModel with this model path for encode_queries and encode_corpus (see FlagEmbedding).
Gemma2Model), used as an encoder: input text → last-token hidden state → L2-normalized embedding.hidden_size=3584, num_hidden_layers=42, num_attention_heads=16, num_key_value_heads=8, intermediate_size=14336, head_dim=256, max_position_embeddings=8192, sliding_window=4096, vocab_size=256002 (includes special tokens <instruct>, <query>).<instruct>, (see / in the model dir).<query>special_tokens_map.jsonadded_tokens.json