Menus Subscribe Search

Follow us


Quick Studies

Figure skater Fontana from Italy performs during the "2010 All That Skate Summer" ice show in Goyang

Figure skater Silvia Fontana from Italy. (Photo: queenyuna/Flickr)

Can You Learn to Judge Creativity?

• April 04, 2014 • 9:26 AM

Figure skater Silvia Fontana from Italy. (Photo: queenyuna/Flickr)

A new study suggests that, with training, amateurs can judge the level of creativity of artwork much like experts would. But is expert opinion always correct?

Once every four years, Americans gather around their television screens to partake in a national pastime, delivering brash, passionate opinions on a topic they barely understand—Olympic figure skating.

Of all the Olympic sports, figure skating is particularly attractive for the armchair judge because the scoring is more subjective than measuring a stick or declaring that a ball went into the net. Artistry ostensibly garners more points than a lifeless, mechanical performance.

Judging creativity, especially in high-stakes situations like the Olympics, is a contentious topic. Who is truly qualified to judge the creativity of a work of art or performance? And can you, as a casual, quadrennial ice-skating observer, ever rise to the ranks of a professional Olympic figure-skating judge?

A new research paper suggests that amateurs can, indeed, be trained to be better judges of creativity—at least when it comes to children’s paintings.

Study participants in the training group were given a brief lesson about the “subcomponents” of creativity, as determined by previous research. The training group participated in a practice round, where it rated the creativity level of paintings from one (not creative at all) to seven (very creative). Afterwards, the participants were told the “actual” creativity level, provided by a panel of expert judges.

“Non-trained judges seemed to agree on something else than the real creativity of the drawings.”

The control group, on the other hand, performed an unrelated task with the set of paintings. These subjects did not learn about the components of creativity.

The results showed that participants in the training group were more likely to excel at judging the creativity of children’s paintings in two distinct ways.

First, those who received training were more likely than the control group to deliver similar creativity ratings as the panel of expert judges.

Additionally, when the training group returned to the lab four weeks later to rate the same exact paintings, it tended to be more reliable, rating the paintings the same as it had in the first trial. The control group was not as reliable with its ratings.

Interestingly enough, while the control group participants’ ratings did not match the experts’, they did tend to agree with one another. In fact, they agreed with each other just as much as the experts and the training group. The authors write, “Non-trained judges seemed to agree on something else than the real creativity of the drawings.”

One mitigating factor, they hypothesize, could be that the group of participants was relatively homogenous—mostly young female university students.

The researchers say there’s no way to know what the untrained group was measuring. They emphasize that agreement with expert opinion should be the baseline of true creativity—after all, they do research creativity for a living.

Still, four years from now when you’re loudly debating the lack of spark in Ashley Wagner’s triple lutz—fifth beer of the night in hand—it’s good to know that while your opinion might not vibe with the experts, it’ll likely be the same as that of your barroom compatriots.

Bettina Chang
Associate Digital Editor Bettina Chang previously directed editorial content at HomeStyle and Real Estate Weekly. A Chicago native, she serves on the board of directors for Supplies for Dreams, working to improve education outcomes for Chicago Public Schools students. Follow her on Twitter @bechang8.

More From Bettina Chang

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

October 17 • 4:00 PM

What All Military Families Need to Know About High-Cost Lenders

Lessons from over a year on the beat.


October 17 • 2:00 PM

The Majority of Languages Do Not Have Gendered Pronouns

A world without “he.” Or “she.”


October 17 • 11:01 AM

How to Water a Farm in Sandy Ground

Physicists investigate how to grow food more efficiently in fine-grained soil.


October 17 • 10:00 AM

Can Science Fiction Spur Science Innovation?

Without proper funding, the answer might not even matter.


October 17 • 8:00 AM

Seattle, the Incredible Shrinking City

Seattle is leading the way in the micro-housing movement as an affordable alternative to high-cost city living.


October 17 • 6:00 AM

‘Voodoo Death’ and How the Mind Harms the Body

Can an intense belief that you’re about to die actually kill you? Researchers are learning more about “voodoo death” and how it isn’t limited to superstitious, foreign cultures.


October 17 • 4:00 AM

That Arts Degree Is Paying Off

A survey of people who have earned degrees in the arts find they are doing relatively well, although their education didn’t provide much guidance on managing a career.


October 16 • 4:00 PM

How (Some) Economists Are Like Doomsday Cult Members

Cognitive dissonance and clinging to paradigms even in the face of accumulated anomalous facts.


October 16 • 2:00 PM

The Latest—and Most Mysterious—Player in the Nasty Battle Over Net Neutrality

As the FCC considers how to regulate Internet providers, the telecom industry’s stealth campaign for hearts and minds encompasses everything from art installations to LOLcats.


October 16 • 12:00 PM

How Many Ads Is Too Many Ads?

The conundrum of online video advertising.


October 16 • 11:00 AM

Unlocking Consciousness

A study of vegetative patients closes in on the nature of consciousness.


October 16 • 10:00 AM

The False Promises of Higher Education

Danielle Henderson spent six years and $60,000 on college and beyond. The effects of that education? Not as advertised.


October 16 • 8:00 AM

Faster Justice, Closer to Home: The Power of Community Courts

Community courts across the country are fighting judicial backlog and lowering re-arrest rates.


October 16 • 6:00 AM

Killing Your Husband to Save Yourself

Without proper legal instruments, women with abusive partners are often forced to make a difficult choice: kill or be killed.


October 16 • 4:00 AM

Personality Traits Linked to Specific Diseases

New research finds neurotic people are more likely to suffer a serious health problem.


October 16 • 2:00 AM

Comparing Apples to the Big Apple: Yes, Washington, D.C., Is More Expensive Than New York City

Why shouldn’t distant locales tied to jobs in the urban core count in a housing expenditure study?


October 15 • 4:00 PM

Why Asian American Parents Are the Least Likely to Spank Their Kids

Highly educated, middle-class parents are less likely to use corporal punishment to discipline their children than less-educated, working-class, and poor parents.


October 15 • 2:00 PM

The Federal Government’s New Doctor Payments Website Is Worthy of a Recall

Charles Ornstein takes a test drive using the federal government’s new website for drug and device payments and finds it virtually unusable.


October 15 • 12:00 PM

How Cosmetic Companies Get Away With Pseudoscience

Anti-aging creams make absurd claims that they repair DNA damage or use stem-cell treatments. When cosmetics companies and dermatologists partner to maximize profits, who is responsible for protecting the consumer?


October 15 • 10:00 AM

What Big Data Can Tell Us About the Things We Eat

Pizza might be the only thing that can bring men and women together.


October 15 • 9:04 AM

‘Looking’ at Art in the Smartphone Age

Technology is a great way to activate gallery space, but it shouldn’t take it over.


October 15 • 8:00 AM

A Brief History of High Heels

How what was once standard footwear for 16th-century Persian horsemen became “fashion’s most provocative accessory.”


October 15 • 7:22 AM

Advice for Emergency Alert Systems: Don’t Cry Wolf

A survey finds college students don’t always take alerts seriously.


October 15 • 6:00 AM

The Battle Over High School Animal Dissection

Is the biology class tradition a useful rite of passage or a schoolroom relic?


October 15 • 4:00 AM

Green Surroundings Linked to Higher Student Test Scores

New research on Massachusetts schoolchildren finds a tangible benefit to regular exposure to nature.


Follow us


How to Water a Farm in Sandy Ground

Physicists investigate how to grow food more efficiently in fine-grained soil.

Unlocking Consciousness

A study of vegetative patients closes in on the nature of consciousness.

Advice for Emergency Alert Systems: Don’t Cry Wolf

A survey finds college students don't always take alerts seriously.

Brain’s Reward Center Does More Than Manage Rewards

Nucleus accumbens tracks many different connections in the world, a new rat study suggests.

A City’s Fingerprints Lie in Its Streets and Alleyways

Researchers propose another way to analyze the character and evolution of cities.

The Big One

One company, Amazon, controls 67 percent of the e-book market in the United States—down from 90 percent five years ago. September/October 2014 new-big-one-5

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.