A side-by-side test run by the account Design Arena found that five AI judges preferred a web design containing nearly four times as much text as its minimalist rival, while the human who made the request chose the minimalist option on the right.
Design evaluation test · LLM-as-a-judge
AI judges chose the busier website. The human chose the minimalist one.
In a side-by-side test using the same request, five AI judges each picked the design with nearly four times as much text. The person who made the request preferred the sparse design.
5 of 5
AI judges selected the busier design
~4×
More text on the winning design
1 of 1
Human requester chose the minimalist design
Text volume: minimalist vs. busier design
Each block = one unit of text volume (busier design shown at nearly 4×)
The AI judges favored the four-block design. The human favored the one-block design.
Preference tally: AI judges vs. human requester
Each block = one preference vote
Why it matters
AI systems are increasingly used to grade and rank generated output, including websites. If judges favor density, developers who optimize for their scores may optimize for clutter rather than quality.
Limits of the test
This is a single, informal comparison with one request and no published methodology or independent verification. It illustrates a possible bias rather than establishing a benchmark.
Takeaway
Test AI judges against human preferences before trusting their rankings.
Judges may reward surface features such as length and density.
A human requester's stated preference diverged from the machine raters.
Teams using AI to judge design output should validate judges on their own data.
Continue reading The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.
Already purchased? Sign in ✓ Signed in — this article isn’t included in your current plan.Unlocking the full article...