Menus Subscribe Search

Findings

research-papers

(Photo: Protasov AN/Shutterstock)

Linguistic Analysis Reveals Research Fraud

• August 28, 2014 • 4:00 AM

(Photo: Protasov AN/Shutterstock)

An examination of papers by the discredited Diederik Stapel finds linguistic differences between his legitimate and fraudulent studies.

Does the name Diederik Stapel ring a bell? He’s the prominent Dutch psychologist who, in 2011, was found to have engaged in research fraud on a massive scale. Much of its data, it now appears, was simply made up.

Could we—should we—have realized his too-good-to-be-true findings were, in fact, fiction? More importantly, can we spot the next guy whose provocative assertions are based on fraudulent data?

It’s a difficult task, to be sure. But David Markowitz and Jeffrey Hancock of Cornell University report that, in Stapel’s case, they were able to classify his research as legitimate or fraudulent “with above-chance accuracy” through careful linguistic analysis.

“Words such as ‘profoundly,’ ‘extremely’ and ‘considerably’ frame the (false) findings as having a substantial and dramatic impact.”

Looking at 49 papers he authored—24 fraudulent, 25 legitimate—they found telltale differences in his writing that indicated, at least to an extent, whether the research was real. They conclude that, even in “highly edited” scientific papers, “deception can be revealed.”

Using Wmatrix, a tool that provides linguistic analysis by investigating such variables as word frequency and grammar, Markowitz and Hancock found Stapel’s writing style varied in several ways when he described fake, rather than genuine, data.

Tellingly, “Stapel used nearly 3,000 fewer adjectives in his fake papers than his genuine papers,” they write in the online journal PLoS One. This pattern is consistent with the theory that “descriptive recalls of real experiences are more sensory and contextually driven.”

“Stapel also wrote with more certainty when describing his take data,” the researchers add, “using nearly one-third more certainty terms than he did in the genuine articles. Words such as ‘profoundly,’ ‘extremely’ and ‘considerably’ frame the (false) findings as having a substantial and dramatic impact.”

In other words, when the results were real, he didn’t feel the need to be quite so emphatic about their importance.

The number of experiments and references per paper did not differ significantly between the real and fake studies. However, the fraudulent papers had fewer authors, on average, than the genuine ones—no surprise, as it is “typically easier to deceive in the presence of a smaller group.”

Of course, other deceptive researchers may not leave precisely the same linguistic clues. But this study provides evidence that, even when dealing with the often-dry, heavily edited world of research papers, “language cues are important in deception detection.”

Markowitz and Hancock’s paper, by the way, is available free online. Feel free to count the number of adjectives they use.

Tom Jacobs
Staff writer Tom Jacobs is a veteran journalist with more than 20 years experience at daily newspapers. He has served as a staff writer for The Los Angeles Daily News and the Santa Barbara News-Press. His work has also appeared in The Los Angeles Times, Chicago Tribune, and Ventura County Star.

More From Tom Jacobs

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

September 19 • 10:00 AM

Why the Poor Remain Poor

A follow-up to “How Being Poor Makes You Poor.”


September 19 • 9:03 AM

Why Science Won’t Defeat Ebola

While science will certainly help, winning the battle against Ebola is a social challenge.


September 19 • 8:00 AM

Burrito Treason in the Lone Star State

Did Meatless Mondays bring down Texas Agriculture Commissioner Todd Staples?


September 19 • 7:31 AM

Savor Good Times, Get Through the Bad Ones—With Categories

Ticking off a category of things to do can feel like progress or a fun time coming to an end.


September 19 • 6:00 AM

The Most Untouchable Man in Sports

How the head of the governing body for the world’s most popular sport freely wields his wildly incompetent power.


September 19 • 4:00 AM

The Danger of Dining With an Overweight Companion

There’s a good chance you’ll eat more unhealthy food.



September 18 • 4:00 PM

Racial Disparity in Imprisonment Inspires White People to Be Even More Tough on Crime

White Americans are more comfortable with punitive and harsh policing and sentencing when they imagine that the people being policed and put in prison are black.



September 18 • 2:00 PM

The Wages of Millions Are Being Seized to Pay Past Debts

A new study provides the first-ever tally of how many employees lose up to a quarter of their paychecks over debts like unpaid credit card or medical bills and student loans.


September 18 • 12:00 PM

When Counterfeit and Contaminated Drugs Are Deadly

The cost and the crackdown, worldwide.


September 18 • 10:00 AM

How Do You Make a Living, Molly Crabapple?

Noah Davis talks to Molly Crapabble about Michelangelo, the Medicis, and the tension between making art and making money.


September 18 • 9:00 AM

Um, Why Are These Professors Creeping on My Facebook Page?

The ethics of student-teacher “intimacy”—on campus and on social media.


September 18 • 8:00 AM

Welcome to the Economy Economy

With the recent introduction of Apple Pay, the Silicon Valley giant is promising to remake how we interact with money. Could iCoin be next?



September 18 • 6:09 AM

How to Build a Better Election

Elimination-style voting is harder to fiddle with than majority rule.


September 18 • 6:00 AM

Homeless on Purpose

The latest entry in a series of interviews about subculture in America.


September 18 • 4:00 AM

Why Original Artworks Move Us More Than Reproductions

Researchers present evidence that hand-created artworks convey an almost magical sense of the artist’s essence.


September 17 • 4:00 PM

Why Gun Control Groups Have Moved Away From an Assault Weapons Ban

A decade after the ban expired, gun control groups say that focusing on other policies will save more American lives.


September 17 • 2:00 PM

Can You Make Two People Like Each Other Just By Telling Them That They Should?

OKCupid manipulates user data in an attempt to find out.


September 17 • 12:00 PM

Understanding ISIL Messaging Through Behavioral Science

By generating propaganda that taps into individuals’ emotional and cognitive states, ISIL is better able motivate people to join their jihad.


September 17 • 10:00 AM

Pulling Punches: Why Sports Leagues Treat Most Offenders With Leniency

There’s a psychological explanation for the weak punishment given to Ray Rice before a video surfaced that made a re-evaluation unavoidable.


September 17 • 9:44 AM

No Innovation Without Migration: Portlandia Is Dying

Build an emerald city. Attract the best and brightest with glorious amenities. They will come and do nothing.



September 17 • 8:00 AM

Why Don’t We Have Pay Toilets in America?

Forty years ago, thanks to an organization founded by four high school friends, human rights beat out the free market—and now we can all pee for free.


Follow us


Savor Good Times, Get Through the Bad Ones—With Categories

Ticking off a category of things to do can feel like progress or a fun time coming to an end.

How to Build a Better Election

Elimination-style voting is harder to fiddle with than majority rule.

Do Conspiracy Theorists Feed on Unsuspecting Internet Trolls?

Not literally, but debunkers and satirists do fuel conspiracy theorists' appetites.

3-D Movies Aren’t That Special

Psychologists find that 3-D doesn't have any extra emotional impact.

To Protect Against Meltdowns, Banks Must Map Financial Interconnections

A new model suggests looking beyond balance sheets, studying the network of investment as well.

The Big One

One in three drivers in Brooklyn's Park Slope—at certain times of day—is just looking for parking. The same goes for drivers in Manhattan's SoHo. September/October 2014 new-big-one-3

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.