from typing import Anyimport torch from huggingface_hub import hf_hub_downloadfrom transformers import Mistral3ForConditionalGenerationfrom transformers import AutoTokenizer def load_system_prompt(repo_id: str, filename: str) -> dict[str, Any]: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() index_begin_think = system_prompt.find("[THINK]") index_end_think = system_prompt.find("[/THINK]") return { "role": "system", "content": [ {"type": "text", "text": system_prompt[:index_begin_think]}, { "type": "thinking", "thinking": system_prompt[ index_begin_think + len("[THINK]") : index_end_think ], "closed": True, }, { "type": "text", "text": system_prompt[index_end_think + len("[/THINK]") :], }, ], } model_id = "mistralai/Magistral-Small-2509" tokenizer = AutoTokenizer.from_pretrained(model_id, tokenizer_type="mistral")model = Mistral3ForConditionalGeneration.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto").eval() SYSTEM_PROMPT = load_system_prompt(model_id, "SYSTEM_PROMPT.txt")image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438"messages = [ SYSTEM_PROMPT, { "role": "user", "content": [ { "type": "text", "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.", }, {"type": "image_url", "image_url": {"url": image_url}}, ], },] tokenized = tokenizer.apply_chat_template(messages, return_dict=True) input_ids = torch.tensor(tokenized.input_ids, device="cuda").unsqueeze(0)attention_mask = torch.tensor(tokenized.attention_mask, device="cuda").unsqueeze(0)pixel_values = torch.tensor( tokenized.pixel_values[0], dtype=torch.bfloat16, device="cuda").unsqueeze(0)image_sizes = torch.tensor(pixel_values.shape[-2:], device="cuda").unsqueeze(0) with torch.inference_mode(): output = model.generate( input_ids=input_ids, attention_mask=attention_mask, pixel_values=pixel_values, image_sizes=image_sizes, )[0] decoded_output = tokenizer.decode( output[ len(tokenized.input_ids) : ( -1 if output[-1] == tokenizer.eos_token_id else len(output) ) ])print(decoded_output)# [THINK]Alright, let's analyze the image carefully. It's a scene from a Pokémon game. The player is controlling Pikachu, which is at level 42 with full HP (83/83). The opponent is a Pidgey at level 17. The question is asking what action the player should take in this situation. # First, let's list all the possible actions available. From the bottom of the screen, the options are:# 1. FIGHT# 2. BAG# 3. POKÉMON# 4. RUN # Now, let's consider each option: # 1. **FIGHT**: This means using Pikachu's moves to attack the Pidgey.# - Pros: Pikachu is at a higher level (42) compared to Pidgey (17), so it has a significant advantage. Pikachu's HP is full, so it's in good condition to fight. Fighting could potentially win the battle quickly.# - Cons: Even though Pikachu is stronger, there's always a risk of Pidgey landing a lucky hit or using a powerful move. However, given the level difference, this is less likely. # 2. **BAG**: This means using items from the bag to help in the battle.# - Pros: Could use a potion to heal (though Pikachu is already at full HP), or use another item like a Poké Ball to try and catch Pidgey.# - Cons: Using items might be less efficient than just fighting, especially since Pikachu is already at full health. Also, if the goal is to catch Pidgey, using items to weaken it first might be useful, but the immediate advantage isn't clear. # 3. **POKÉMON**: This means switching to another Pokémon from the team.# - Pros: If the player has another Pokémon that is stronger or has moves that are super effective against Pidgey, this could be useful.# - Cons: Pikachu is already at a significant level advantage and is at full health, so switching might not be necessary unless there's a strategic reason (e.g., leveling up another Pokémon). # 4. **RUN**: This means attempting to flee from the battle.# - Pros: If the player wants to avoid the battle for some reason (e.g., saving time, or wanting to catch Pidgey without weakening it), running could be useful.# - Cons: If the goal is to catch or defeat Pidgey, running would not help in achieving that. Also, running might not always be successful, especially in a grass encounter where the chance to flee might be lower. # Now, considering the context, if the goal is to defeat Pidgey, the best action would be to FIGHT since Pikachu is at a significant advantage in terms of level and health. Using the BAG could be useful if the goal is to catch Pidgey, as items like Poké Balls or berries could be used. Switching Pokémon doesn't seem necessary given Pikachu's current status. Running would only be useful if the player wants to avoid the battle, which seems unlikely if the goal is to progress in the game. # However, the question asks for the best action without specifying the goal. Assuming the goal is to defeat Pidgey efficiently, the best action would be to FIGHT. # But to list all possible actions and their pros and cons: # 1. **FIGHT**# - Pros: High chance of winning quickly due to level and health advantage.# - Cons: Small risk of Pikachu getting damaged. # 2. **BAG**# - Pros: Could use items to catch Pidgey or support Pikachu (though not necessary for healing).# - Cons: Might be less efficient if the goal is just to defeat Pidgey. # 3. **POKÉMON**# - Pros: Could switch to another Pokémon for strategic reasons (e.g., type advantage, training a different Pokémon).# - Cons: Pikachu is already at a strong position, so switching might be unnecessary unless there's another goal. # 4. **RUN**# - Pros: Escape the battle if needed.# - Cons: Does not contribute to progress if the goal is to defeat or catch Pidgey. # Given that, if the goal is to defeat Pidgey, the best action is to FIGHT. If the goal is to catch Pidgey, using the BAG to throw a Poké Ball could be the best action.[/THINK]Here are the possible actions and their explanations: # 1. **FIGHT**# - **Pros**: Pikachu is at a much higher level (42) compared to Pidgey (17) and has full HP. This makes it very likely to win quickly.# - **Cons**: There's always a small risk that Pidgey could land a lucky hit, but this is unlikely given the level difference. # 2. **BAG**# - **Pros**: You could use items to catch Pidgey (e.g., Poké Ball) or support Pikachu (though Pikachu is already at full HP).# - **Cons**: Using items might be less efficient if the goal is simply to defeat Pidgey, as fighting directly could be quicker. # 3. **POKÉMON**# - **Pros**: You could switch to another Pokémon that might have a type advantage or that you want to train.# - **Cons**: Pikachu is already in a strong position to defeat Pidgey, so switching might not be necessary unless there's another strategic reason. # 4. **RUN**# - **Pros**: You can escape the battle if you need to, for example, if you want to preserve Pikachu's health for a tougher battle ahead.# - **Cons**: Running doesn't help you progress if your goal is to defeat or catch Pidgey. Additionally, the success rate for running might be lower in a grass encounter. # Given these considerations, if your goal is to defeat Pidgey, the best action is likely to **FIGHT**, as Pikachu is at a significant advantage. If your goal is to catch Pidgey, using the **BAG** to throw a Poké Ball could be the best choice. If you're looking to train a different Pokémon, you might consider switching with **POKÉMON**, and if you need to preserve resources or Pikachu's health, **RUN** could be an option.