treadon

prompt-fungineer-v2

README

License: apache-2.0

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "treadon/prompt-fungineer-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)

def expand_prompt(short_description: str, max_new_tokens: int = 256) -> str:
    messages = [
        {
            "role": "user",
            "content": f"Expand this into a detailed image generation prompt: {short_description}"
        }
    ]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt").to(model.device)

    with torch.no_grad():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=0.7,
            top_p=0.9,
        )

    generated = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    return generated.strip()

# Example
import torch
print(expand_prompt("cherry blossoms in rain"))

Example output:

A cherry blossom tree in full bloom photographed at the moment of rain, the petals floating and falling in the downpour. Fuji X-T5, 24mm f/4 ultra-wide lens, f/8, ISO 800, 1/1000s. The petals are translucent pink and white, each one catching the raindrop light differently. The rain falls in the background as a soft white blur. Color: vivid pink blossom against white rain, the sky behind in soft grey. Matte finish. The delicate pink against the wet grey sky. Japanese seasonal beauty.

Model Details

Table with columns: Property, Value
Property	Value
Base model	Qwen/Qwen3-0.6B (2025 architecture)
Parameters	596M
Training method	Full fine-tune (all parameters unfrozen)
Training data	9,400 Claude-generated prompt pairs
Categories	11 (landscapes, portraits, sci-fi, abstract, architecture, animals, historical, food, macro, emotions, vehicles)
Style distribution	5 styles (photographic 52%, cinematic 14%, painterly 14%, digital art 10%, illustration 9%)
Epochs	4
Learning rate	2e-5, cosine schedule with 6% warmup
Effective batch size	32 (batch 1 x gradient accumulation 32)
Precision	float32
Gradient checkpointing	Enabled
Max sequence length	384 tokens
Training time	~2h 50m on Apple M4 Pro (MPS backend)
Final train loss	1.51
Final eval loss	2.67
Optimizer	AdamW (weight decay 0.01)
License	Apache 2.0

v1 vs v2 Comparison

Prompt Fungineer v2 generates longer, more specific, and more technically detailed prompts compared to v1.

Table with columns: Metric, v1 (GPT-2 355M), v2 (Qwen3-0.6B)
Metric	v1 (GPT-2 355M)	v2 (Qwen3-0.6B)
Parameters	355M	596M (1.7x)
Architecture	GPT-2 (2019)	Qwen3 (2025)
Avg output length	92 words	117 words (+27%)
Avg generation time	2.6s	4.8s
Training data	Web-scraped prompts	9,400 Claude-generated pairs

Side-by-Side Examples

Table with columns: Input, v1 (GPT-2 355M), v2 (Qwen3-0.6B)
Input	v1 (GPT-2 355M)	v2 (Qwen3-0.6B)
"sunset over the ocean"	Generic: "A breathtaking image of a tranquil, calm image of the horizon at sunset, revealing a serene, crystal-clear ocean..."	Specific: "The Pacific Ocean at sunset, the sea surface in perfect mirror -- the sky above reflected in the dark water below, the sky and its reflection creating a continuous white and gold spectacle from horizon to zenith."
"neon-lit street at night"	Generic: "A captivating image of a bustling, neon-lit street at night, with pedestrians weaving through..."	Specific: "A city street at 2am, every surface lit by a different coloured neon sign -- red traffic lights, blue bus stop signs, green bus number boards, yellow taxi stop boards."
"cherry blossoms in rain"	Wrong scene: confused with "a lush, tropical rainforest"	Correct scene: cherry blossom tree in rain, named specific camera (Fuji X-T5, 24mm f/4), technical settings (f/8, ISO 800, 1/1000s)
"wolf howling at the moon"	Hallucinated: "A haunting image of a deer stalking the night sky"

Key Improvements in v2

Specificity: v2 names concrete details (locations, camera models, times of day) instead of generic adjectives
Technical vocabulary: v2 includes camera specs (f-stops, ISO, lens mm), composition directions, and lighting terms
Subject accuracy: v2 stays true to the input description; v1 sometimes drifted to unrelated subjects
Scene coherence: v2 builds a unified scene with a consistent mood rather than stacking unrelated descriptors

Known Limitations

Chat template artifacts: v2 occasionally includes fragments of the chat template (e.g., user, assistant markers) in the generated output. Post-processing to strip these is recommended.
Hallucinated details: v2 can introduce incorrect details. For example, "old man playing chess" produced a prompt mentioning a "mahjong table" despite correctly describing chess pieces and a chessboard.
Slower generation: v2 averages 4.8s per prompt vs 2.6s for v1, due to the larger model and longer outputs.
Repetitive phrasing: v2 sometimes falls into repetitive philosophical loops at the end of longer outputs (e.g., "The moon that is watching and the animal that is watching the moon. The moon that is always there.").
Overfitting gap: The gap between train loss (1.51) and eval loss (2.67) suggests some overfitting, which may contribute to occasional repetition or formulaic outputs.

Training Data

The model was trained on treadon/prompt-fungineer-v2-training-data, a dataset of 9,400 prompt pairs generated by Claude (Sonnet/Opus).

Category distribution:

Table with columns: Category, Count
Category	Count
landscapes_nature	1,288
portraits_people	1,091
architecture_urban	979
historical_vintage	940
emotions_concepts	812
abstract_artistic	774
food_stilllife	750
macro_closeup	742
animals_wildlife	701

Each pair consists of a short input (1-9 words, avg 4.8) and a detailed output (53-181 words, avg 90.5). The data spans 5 style profiles: photographic, cinematic, painterly, digital art, and illustration.

Quality controls applied during generation:

No copyrighted names or characters
No NSFW content
Varied sentence structures
Prompt lengths between 50-150 words
Each pair validated: input at most 15 words, output at least 20 words

Training Infrastructure

Trained on an Apple M4 Pro with 64GB unified memory using the MPS backend. Key lessons from training on Apple Silicon:

Use HuggingFace Trainer, not raw PyTorch loops (MPS shader compilation can hang with manual device placement)
Use float32 (BF16 training on MPS causes silent hangs or incorrect results)
Use batch_size=1 + gradient accumulation (float32 activations for 28-layer models blow up memory at larger batch sizes)
Enable gradient checkpointing (trades ~30% more compute for ~70% less activation memory)

Citation

bibtex
@misc{prompt-fungineer-v2,
  author = {Ritesh Khanna},
  title = {Prompt Fungineer v2: Distilling Prompt Engineering Into a Small Local Model},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/treadon/prompt-fungineer-v2}
}

Predecessor

treadon/prompt-fungineer-355M -- GPT-2 Medium (355M), trained in 2023 on web-scraped prompt data.

More from me

For other projects and writeups, see riteshkhanna.com, follow @treadon on X, or treadon on Hugging Face.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

treadon

Model Tree

Base

Qwen/Qwen3-0.6B

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "treadon/prompt-fungineer-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)

def expand_prompt(short_description: str, max_new_tokens: int = 256) -> str:
    messages = [
        {
            "role": "user",
            "content": f"Expand this into a detailed image generation prompt: {short_description}"
        }
    ]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt").to(model.device)

    with torch.no_grad():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=0.7,
            top_p=0.9,
        )

    generated = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    return generated.strip()

# Example
import torch
print(expand_prompt("cherry blossoms in rain"))

Example output:

A cherry blossom tree in full bloom photographed at the moment of rain, the petals floating and falling in the downpour. Fuji X-T5, 24mm f/4 ultra-wide lens, f/8, ISO 800, 1/1000s. The petals are translucent pink and white, each one catching the raindrop light differently. The rain falls in the background as a soft white blur. Color: vivid pink blossom against white rain, the sky behind in soft grey. Matte finish. The delicate pink against the wet grey sky. Japanese seasonal beauty.

Model Details

Table with columns: Property, Value
Property	Value
Base model	Qwen/Qwen3-0.6B (2025 architecture)
Parameters	596M
Training method	Full fine-tune (all parameters unfrozen)
Training data	9,400 Claude-generated prompt pairs
Categories	11 (landscapes, portraits, sci-fi, abstract, architecture, animals, historical, food, macro, emotions, vehicles)
Style distribution	5 styles (photographic 52%, cinematic 14%, painterly 14%, digital art 10%, illustration 9%)
Epochs	4
Learning rate	2e-5, cosine schedule with 6% warmup
Effective batch size	32 (batch 1 x gradient accumulation 32)
Precision	float32
Gradient checkpointing	Enabled
Max sequence length	384 tokens
Training time	~2h 50m on Apple M4 Pro (MPS backend)
Final train loss	1.51
Final eval loss	2.67
Optimizer	AdamW (weight decay 0.01)
License	Apache 2.0

v1 vs v2 Comparison

Prompt Fungineer v2 generates longer, more specific, and more technically detailed prompts compared to v1.

Table with columns: Metric, v1 (GPT-2 355M), v2 (Qwen3-0.6B)
Metric	v1 (GPT-2 355M)	v2 (Qwen3-0.6B)
Parameters	355M	596M (1.7x)
Architecture	GPT-2 (2019)	Qwen3 (2025)
Avg output length	92 words	117 words (+27%)
Avg generation time	2.6s	4.8s
Training data	Web-scraped prompts	9,400 Claude-generated pairs

Side-by-Side Examples

Table with columns: Input, v1 (GPT-2 355M), v2 (Qwen3-0.6B)
Input	v1 (GPT-2 355M)	v2 (Qwen3-0.6B)
"sunset over the ocean"	Generic: "A breathtaking image of a tranquil, calm image of the horizon at sunset, revealing a serene, crystal-clear ocean..."	Specific: "The Pacific Ocean at sunset, the sea surface in perfect mirror -- the sky above reflected in the dark water below, the sky and its reflection creating a continuous white and gold spectacle from horizon to zenith."
"neon-lit street at night"	Generic: "A captivating image of a bustling, neon-lit street at night, with pedestrians weaving through..."	Specific: "A city street at 2am, every surface lit by a different coloured neon sign -- red traffic lights, blue bus stop signs, green bus number boards, yellow taxi stop boards."
"cherry blossoms in rain"	Wrong scene: confused with "a lush, tropical rainforest"	Correct scene: cherry blossom tree in rain, named specific camera (Fuji X-T5, 24mm f/4), technical settings (f/8, ISO 800, 1/1000s)
"wolf howling at the moon"	Hallucinated: "A haunting image of a deer stalking the night sky"

Key Improvements in v2

Specificity: v2 names concrete details (locations, camera models, times of day) instead of generic adjectives
Technical vocabulary: v2 includes camera specs (f-stops, ISO, lens mm), composition directions, and lighting terms
Subject accuracy: v2 stays true to the input description; v1 sometimes drifted to unrelated subjects
Scene coherence: v2 builds a unified scene with a consistent mood rather than stacking unrelated descriptors

Known Limitations

Chat template artifacts: v2 occasionally includes fragments of the chat template (e.g., user, assistant markers) in the generated output. Post-processing to strip these is recommended.
Hallucinated details: v2 can introduce incorrect details. For example, "old man playing chess" produced a prompt mentioning a "mahjong table" despite correctly describing chess pieces and a chessboard.
Slower generation: v2 averages 4.8s per prompt vs 2.6s for v1, due to the larger model and longer outputs.
Repetitive phrasing: v2 sometimes falls into repetitive philosophical loops at the end of longer outputs (e.g., "The moon that is watching and the animal that is watching the moon. The moon that is always there.").
Overfitting gap: The gap between train loss (1.51) and eval loss (2.67) suggests some overfitting, which may contribute to occasional repetition or formulaic outputs.

Training Data

The model was trained on treadon/prompt-fungineer-v2-training-data, a dataset of 9,400 prompt pairs generated by Claude (Sonnet/Opus).

Category distribution:

Table with columns: Category, Count
Category	Count
landscapes_nature	1,288
portraits_people	1,091
architecture_urban	979
historical_vintage	940
emotions_concepts	812
abstract_artistic	774
food_stilllife	750
macro_closeup	742
animals_wildlife	701

Quality controls applied during generation:

No copyrighted names or characters
No NSFW content
Varied sentence structures
Prompt lengths between 50-150 words
Each pair validated: input at most 15 words, output at least 20 words

Training Infrastructure

Trained on an Apple M4 Pro with 64GB unified memory using the MPS backend. Key lessons from training on Apple Silicon:

Use HuggingFace Trainer, not raw PyTorch loops (MPS shader compilation can hang with manual device placement)
Use float32 (BF16 training on MPS causes silent hangs or incorrect results)
Use batch_size=1 + gradient accumulation (float32 activations for 28-layer models blow up memory at larger batch sizes)
Enable gradient checkpointing (trades ~30% more compute for ~70% less activation memory)

Citation

bibtex
@misc{prompt-fungineer-v2,
  author = {Ritesh Khanna},
  title = {Prompt Fungineer v2: Distilling Prompt Engineering Into a Small Local Model},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/treadon/prompt-fungineer-v2}
}

Predecessor

treadon/prompt-fungineer-355M -- GPT-2 Medium (355M), trained in 2023 on web-scraped prompt data.

More from me

For other projects and writeups, see riteshkhanna.com, follow @treadon on X, or treadon on Hugging Face.