GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
center-of-excellence
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
center-of-excellence
Model Tree
TinyLlama/TinyLlama-1.1B-Chat-v1.0
Input Modalities
Output Modalities
Supported Functionality
Evaluated on a 75-example held-out test set:
| Metric | Score |
|---|---|
| Overall MAE | 0.51 |
| Accuracy (within 1 point) | 82.1% |
| Accuracy (within 2 points) | 91.0% |
| Parse success rate | 89.3% |
Inference speed: ~3.1s on Apple Silicon (MPS), ~6.2s on CPU.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelimport json model_repo = "center-of-excellence/extract-prompt-quality-criteria"base_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0" # Auto-detect best devicedevice = "cuda" if torch.cuda.is_available() else ("mps" if torch.backends.mps.is_available() else "cpu")print(f"Using device: {device}") # Load tokenizertokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)if tokenizer.pad_token is None: tokenizer.pad_token = tokenizer.eos_token # Load model — merge LoRA weights for faster inferencebase_model = AutoModelForCausalLM.from_pretrained( base_model_name, dtype=torch.float16 if device in ("cuda", "mps") else torch.float32, trust_remote_code=True, low_cpu_mem_usage=True)model = PeftModel.from_pretrained(base_model, model_repo)model = model.merge_and_unload() # merge for faster inferencemodel = model.to(device)model.eval() # Analyze a promptprompt = "Classify text into categories" instruction = """You are a prompt quality analyzer. Analyze the given prompt and extract quality scores for each criterion. Criteria (score 1-10):1. clarity_and_specificity: Are instructions clear and specific?2. context_sufficiency: Is sufficient background/context provided?3. examples_provided: Are concrete examples included?4. output_format_specification: Is the expected output format clearly defined?5. edge_case_handling: Are edge cases and exceptions addressed?6. tone_and_style_guidance: Is communication style/tone specified?7. constraint_definition: Are limits and boundaries clearly set?8. relevance_of_examples: Do examples match the domain/task? Respond with ONLY a JSON object containing the scores.""" formatted = f"""<|system|>{instruction}<|end|><|user|>Analyze this prompt: {prompt}<|end|><|assistant|>""" inputs = tokenizer(formatted, return_tensors="pt", truncation=True, max_length=512).to(device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=300, temperature=0.3, do_sample=True, top_p=0.9, repetition_penalty=1.1, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id ) input_length = inputs['input_ids'].shape[1]response = tokenizer.decode(outputs[0][input_length:], skip_special_tokens=True) # Parse JSONstart = response.find('{')end = response.rfind('}') + 1result = json.loads(response[start:end])print(json.dumps(result, indent=2))python
import pandas as pdfrom prompt_quality_transformer import PromptQualityTransformer df = pd.DataFrame({'prompt': ['Analyze the text.', 'Classify sentiment. Return JSON.']}) transformer = PromptQualityTransformer( model_repo="center-of-excellence/extract-prompt-quality-criteria", batch_size=8, use_quantization=True, # int8 for faster GPU inference verbose=True)df_scored = transformer.fit_transform(df)print(df_scored[['prompt', 'overall_score', 'quality_tier']])| Criterion | Description |
|---|---|
| Clarity & Specificity | Are instructions clear and unambiguous? |
| Context Sufficiency | Is enough background provided? |
| Examples Provided | Are there concrete examples? |
| Output Format | Is the expected format specified? |
| Edge Case Handling | Are exceptions addressed? |
| Tone & Style | Is communication style defined? |
| Constraint Definition | Are limits clearly set? |
json
{ "scores": { "clarity_and_specificity": 8, "context_sufficiency": 7, "examples_provided": 6, "output_format_specification": 9, "edge_case_handling": 5, "tone_and_style_guidance": 6, "constraint_definition": 7, "relevance_of_examples": 8 }, "overall_score": 7.0, "quality_tier": "good"}<|system|>, <|user|>, <|assistant|> formatedge_case_handling is the weakest criterion (MAE 1.33) — accuracy improves when prompts explicitly mention edge casesbibtex
@misc{extract-prompt-quality-criteria, author = {Gregoire Cattan and Elif Uzun}, title = {Extract Prompt Quality Criteria}, year = {2025}, publisher = {Hugging Face}, url = {https://huggingface.co/center-of-excellence/extract-prompt-quality-criteria}}{ "scores": { "clarity_and_specificity": 8, "context_sufficiency": 7, "examples_provided": 6, "output_format_specification": 9, "edge_case_handling": 5, "tone_and_style_guidance": 6, "constraint_definition": 7, "relevance_of_examples": 8 }, "overall_score": 7.0, "quality_tier": "good"}| Do examples match the task? |