GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
HowieHwong
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
HowieHwong
Model Tree
meta-llama/Llama-3.1-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This model is trained to optimize user prompts based on their interaction history and preferences. Given a user's conversation history and current query, it rewrites the query into a clearer, more specific, and better-structured prompt.
| Parameter | Value |
|---|---|
| r (rank) | 32 |
| lora_alpha | 32 |
| target_modules | all-linear |
| lora_dropout | 0 |
| bias | none |
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel# Load base modelbase_model_id = "meta-llama/Llama-3.1-8B-Instruct"model = AutoModelForCausalLM.from_pretrained(base_model_id,torch_dtype="auto",device_map="auto")tokenizer = AutoTokenizer.from_pretrained(base_model_id)# Load LoRA adaptermodel = PeftModel.from_pretrained(model, "HowieHwong/ppopt")
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel# Load modelbase_model_id = "meta-llama/Llama-3.1-8B-Instruct"model = AutoModelForCausalLM.from_pretrained(base_model_id,torch_dtype="auto",device_map="auto")tokenizer = AutoTokenizer.from_pretrained(base_model_id)model = PeftModel.from_pretrained(model, "HowieHwong/ppopt")# Prepare inputconversation_history = """User: How do I center a div?Assistant: You can use flexbox: display: flex; justify-content: center; align-items: center;User: What about grid?Assistant: With grid: display: grid; place-items: center;"""current_query = "how to make it responsive"prompt = f"""Based on the conversation history and user preferences, optimize the following query into a clearer, more specific prompt.Conversation History:{conversation_history}Current Query: {current_query}Optimized Prompt:"""# Generatemessages = [{"role": "user", "content": prompt}]input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)outputs = model.generate(input_ids,max_new_tokens=256,temperature=0.7,do_sample=True,pad_token_id=tokenizer.eos_token_id)response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)print(response)
If you want to merge the adapter into the base model:
python
merged_model = model.merge_and_unload()merged_model.save_pretrained("merged_ppopt_llama8b")tokenizer.save_pretrained("merged_ppopt_llama8b")
This model is designed for:
This model is released under the Apache 2.0 license.