GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today โ
SakanaAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
SakanaAI
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Use the code below to get started with the model.
python
import torchfrom transformers import AutoModelForVision2Seq, AutoProcessorfrom PIL import Imageimport requestsย ย # 1. load modeldevice = "cuda" if torch.cuda.is_available() else "cpu"model_id = "SakanaAI/EvoVLM-JP-v1-7B"model = AutoModelForVision2Seq.from_pretrained(model_id, torch_dtype=torch.float16)processor = AutoProcessor.from_pretrained(model_id)model.to(device)ย # 2. prepare inputsurl = "https://images.unsplash.com/photo-1694831404826-3400c48c188d?q=80&w=2070&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D"image = Image.open(requests.get(url, stream=True).raw).convert("RGB")# <image> represents the input image. Please make sure to put the token in your text.text = "<image>\nใใฎไฟกๅทๆฉใฎ่ฒใฏไฝ่ฒใงใใ?"messages = [ {"role": "system", "content": "ใใชใใฏๅฝน็ซใคใๅ่ฆใใชใใๆค้ฒใใใฆใใชใใขใทในใฟใณใใงใใไธใใใใ็ปๅใไธใซใ่ณชๅใซ็ญใใฆใใ ใใใ"}, {"role": "user", "content": text},]inputs = processor.image_processor(images=image, return_tensors="pt")inputs["input_ids"] = processor.tokenizer.apply_chat_template( messages, return_tensors="pt")# 3. generateoutput_ids = model.generate(**inputs.to(device))output_ids = output_ids[:, inputs.input_ids.shape[1] :]generated_text = processor.batch_decode(output_ids, skip_special_tokens=True)[0].strip()print(generated_text)# ใใฎไฟกๅทๆฉใฎ่ฒใฏ้ใงใใThis model is provided for research and development purposes only and should be considered as an experimental prototype. It is not intended for commercial use or deployment in mission-critical environments. Use of this model is at the user's own risk, and its performance and outcomes are not guaranteed. Sakana AI shall not be liable for any direct, indirect, special, incidental, or consequential damages, or any loss arising from the use of this model, regardless of the results obtained. Users must fully understand the risks associated with the use of this model and use it at their own discretion.
We would like to thank the developers of the source models for their contributions and for making their work available.
bibtex
@misc{akiba2024evomodelmerge, title = {Evolutionary Optimization of Model Merging Recipes}, author. = {Takuya Akiba and Makoto Shing and Yujin Tang and Qi Sun and David Ha}, year = {2024}, eprint = {2403.13187}, archivePrefix = {arXiv}, primaryClass = {cs.NE}}import torchfrom transformers import AutoModelForVision2Seq, AutoProcessorfrom PIL import Imageimport requestsย ย # 1. load modeldevice = "cuda" if torch.cuda.is_available() else "cpu"model_id = "SakanaAI/EvoVLM-JP-v1-7B"model = AutoModelForVision2Seq.from_pretrained(model_id, torch_dtype=torch.float16)processor = AutoProcessor.from_pretrained(model_id)model.to(device)ย # 2. prepare inputsurl = "https://images.unsplash.com/photo-1694831404826-3400c48c188d?q=80&w=2070&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D"image = Image.open(requests.get(url, stream=True).raw).convert("RGB")# <image> represents the input image. Please make sure to put the token in your text.text = "<image>\nใใฎไฟกๅทๆฉใฎ่ฒใฏไฝ่ฒใงใใ?"messages = [ {"role": "system", "content": "ใใชใใฏๅฝน็ซใคใๅ่ฆใใชใใๆค้ฒใใใฆใใชใใขใทในใฟใณใใงใใไธใใใใ็ปๅใไธใซใ่ณชๅใซ็ญใใฆใใ ใใใ"}, {"role": "user", "content": text},]inputs = processor.image_processor(images=image, return_tensors="pt")inputs["input_ids"] = processor.tokenizer.apply_chat_template( messages, return_tensors="pt")# 3. generateoutput_ids = model.generate(**inputs.to(device))output_ids = output_ids[:, inputs.input_ids.shape[1] :]generated_text = processor.batch_decode(output_ids, skip_special_tokens=True)[0].strip()print(generated_text)# ใใฎไฟกๅทๆฉใฎ่ฒใฏ้ใงใใ@misc{akiba2024evomodelmerge, title = {Evolutionary Optimization of Model Merging Recipes}, author. = {Takuya Akiba and Makoto Shing and Yujin Tang and Qi Sun and David Ha}, year = {2024}, eprint = {2403.13187}, archivePrefix = {arXiv}, primaryClass = {cs.NE}}