Rank
model
Bradley-Terry
Elo
Win Rate
Wins
Matches
1
ai logo
flux-1-pro
1132.271044.940.548299231537197
2
ai logo
flux-1.1-pro
1080.951028.030.52485401933425
3
ai logo
aurora-20-1-25
1031.861011.120.51235435463086
4
ai logo
imagen-3
1016.821005.760.50443799895047
5
ai logo
frames-23-1-25
987.39995.080.50194282390631
6
ai logo
dalle-3
949.89980.960.497070331450790
7
ai logo
stable-diffusion-3
937.19976.070.495894781214494
8
ai logo
midjourney-5.2
891.95958.050.476375751361182

What is "Bradley-Terry"?

The Bradley-Terry ranking model is a probabilistic model used to predict outcomes in pairwise comparisons. It assigns a strength parameter (reported score) to each item, indicating its likelihood of winning against another. See the wikipedia article for mathematical details.

What do we consider as "Overall preference"?

Here we evaluate the model across all criteria and determine which model has the best overall performance.

All results are directly based on feedback from real human raters. The process of how we came out with results is best described in our blog post.

Examples

Visual examples of the annotators’ preferences

Preference
Which image looks better overall?
flux-1-pro_winner
FLUX.1 [pro]
flux-1-pro_winner
Midjourney
Coherence
Which image feels less weird or unnatural for its style when you look closely? I.e. fewer odd or strange-looking objects or elements
flux-1-pro_winner
FLUX.1 [pro]
flux-1-pro_winner
Midjourney
Alignment
Which image is more aligned with and better adheres to the prompt:
A black and white picture of a white man singing a song
flux-1-pro_winner
FLUX.1 [pro]
flux-1-pro_winner
Midjourney