Model Details
- Base model:
black-forest-labs/FLUX.1-dev
- Method: GRPO (group-relative advantage, critic-free) with ODE→SDE sampling
- Reward: frozen Q-Judger (Qwen-Image-Bench, Qwen3.6-27B), structured score
vector over 5 pillars / 56 facets
- Trainable params: 74.7M of 11.98B (0.62%)
- Target modules:
to_q, to_k, to_v, to_out.0 · rank 64, alpha 64
- Developed by: Dinaiym Dubanaeva, Ala-Too International University, Bishkek, Kyrgyzstan
Training
Table with columns: Setting, Value| Setting | Value |
|---|
| GRPO steps | 10 |
| Prompts / step | 1 |
| Samples / prompt (G) | 3 |
| Resolution | 512×512, 6 denoising steps |
| Optimizer | AdamW, lr 1e-4 |
| Clip ε / KL β | 0.2 / 0.04 |
| SDE noise σ | 0.15 |
| Prompts | Qwen-Image-Bench test split (1,000 prompts, dims_en) |
| Hardware | 1× A100 80GB · 670 s/step · 1.86 h |
Policy (12B) and judge (27B) do not co-reside in 80GB, so the phases are
sequenced: each model is moved to CPU while the other occupies the GPU.
Results
Table with columns: Metric, Value| Metric | Value |
|---|
| Q-Judger parse rate | 1.000 (30/30 prompt–image pairs) |
| Gradient norm | 332 – 764 |
| Policy–reference KL | 0 → 256 |
| Mean reward | 32.11 ± 11.62 |
| OLS slope | +1.34/step (r² = 0.12, not significant) |
The per-step reward is confounded by prompt difficulty (one prompt per step),
so it must not be read as a learning curve.
Limitations
- No baseline comparison yet. No claim of improved image quality is made.
- Small scale. 10 steps, G=3, 512×512 — far below the regime where
flow-matching GRPO is known to move benchmarks.
- Evaluator circularity. The judge supplying the reward is also the judge
reporting the score; KL reached 256 in 10 steps, so over-optimization cannot
be ruled out without a held-out metric.
Usage
from diffusers import FluxPipeline
import torch
pipe = FluxPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16
).to("cuda")
pipe.load_lora_weights("thedinaiym/judge-in-the-loop-flux-lora")
image = pipe("a robot walking in Tokyo", num_inference_steps=20).images[0]
Links
Framework
PEFT 0.19.1 · diffusers 0.39.0 · transformers 5.13.0 · torch 2.12.0+cu130