Menus Subscribe Search

Follow us



(Photo: Protasov AN/Shutterstock)

Linguistic Analysis Reveals Research Fraud

• August 28, 2014 • 4:00 AM

(Photo: Protasov AN/Shutterstock)

An examination of papers by the discredited Diederik Stapel finds linguistic differences between his legitimate and fraudulent studies.

Does the name Diederik Stapel ring a bell? He’s the prominent Dutch psychologist who, in 2011, was found to have engaged in research fraud on a massive scale. Much of its data, it now appears, was simply made up.

Could we—should we—have realized his too-good-to-be-true findings were, in fact, fiction? More importantly, can we spot the next guy whose provocative assertions are based on fraudulent data?

It’s a difficult task, to be sure. But David Markowitz and Jeffrey Hancock of Cornell University report that, in Stapel’s case, they were able to classify his research as legitimate or fraudulent “with above-chance accuracy” through careful linguistic analysis.

“Words such as ‘profoundly,’ ‘extremely’ and ‘considerably’ frame the (false) findings as having a substantial and dramatic impact.”

Looking at 49 papers he authored—24 fraudulent, 25 legitimate—they found telltale differences in his writing that indicated, at least to an extent, whether the research was real. They conclude that, even in “highly edited” scientific papers, “deception can be revealed.”

Using Wmatrix, a tool that provides linguistic analysis by investigating such variables as word frequency and grammar, Markowitz and Hancock found Stapel’s writing style varied in several ways when he described fake, rather than genuine, data.

Tellingly, “Stapel used nearly 3,000 fewer adjectives in his fake papers than his genuine papers,” they write in the online journal PLoS One. This pattern is consistent with the theory that “descriptive recalls of real experiences are more sensory and contextually driven.”

“Stapel also wrote with more certainty when describing his take data,” the researchers add, “using nearly one-third more certainty terms than he did in the genuine articles. Words such as ‘profoundly,’ ‘extremely’ and ‘considerably’ frame the (false) findings as having a substantial and dramatic impact.”

In other words, when the results were real, he didn’t feel the need to be quite so emphatic about their importance.

The number of experiments and references per paper did not differ significantly between the real and fake studies. However, the fraudulent papers had fewer authors, on average, than the genuine ones—no surprise, as it is “typically easier to deceive in the presence of a smaller group.”

Of course, other deceptive researchers may not leave precisely the same linguistic clues. But this study provides evidence that, even when dealing with the often-dry, heavily edited world of research papers, “language cues are important in deception detection.”

Markowitz and Hancock’s paper, by the way, is available free online. Feel free to count the number of adjectives they use.

Tom Jacobs
Staff writer Tom Jacobs is a veteran journalist with more than 20 years experience at daily newspapers. He has served as a staff writer for The Los Angeles Daily News and the Santa Barbara News-Press. His work has also appeared in The Los Angeles Times, Chicago Tribune, and Ventura County Star.

More From Tom Jacobs

A weekly roundup of the best of Pacific Standard and, delivered straight to your inbox.

Recent Posts

September 30 • 10:09 AM

Trust Is Waning, and Inequality May Be To Blame

Trust in others and confidence in institutions is declining, while economic inequality creeps up, a new study shows.

September 30 • 8:00 AM

The Psychology of Penmanship

Graphology: It’s all (probably) bunk.

September 30 • 6:00 AM

The Medium Is the Message, 50 Years Later

Five decades on, what can Marshall McLuhan’s Understanding Media tell us about today?

September 30 • 4:00 AM

Grad School’s Mental Health Problem

Navigating the emotional stress of doctoral programs in a down market.

September 29 • 1:21 PM

Conference Call: Free Will Conference

September 29 • 12:00 PM

How Copyright Law Protects Art From Criticism

A case for allowing the copyright on Gone With the Wind to expire.

September 29 • 10:00 AM

Should We Be Told Who Funds Political Attack Ads?

On the value of campaign finance disclosure.

September 29 • 8:00 AM

Searching for a Man Named Penis

A quest to track down a real Penis proves difficult.

September 29 • 6:00 AM

Why Do So Many People Watch HGTV?

The same reason so many people watch NCIS or Law and Order: It’s all a procedural.

September 29 • 4:00 AM

The Link Between Depression and Terrorism

A new study from the United Kingdom finds a connection between depression and radicalization.

September 26 • 4:00 PM

Fast Track to a Spill?

Oil pipeline projects across America are speeding forward without environmental review.

September 26 • 2:00 PM

Why Liberals Love the Disease Theory of Addiction, by a Liberal Who Hates It

The disease model is convenient to liberals because it spares them having to say negative things about poor communities. But this conception of addiction harms the very people we wish to help.

September 26 • 1:21 PM

Race, Trust, and Split-Second Judgments

September 26 • 9:47 AM

Dopamine Might Be Behind Impulsive Behavior

A monkey study suggests the brain chemical makes what’s new and different more attractive.

September 26 • 8:00 AM

A Letter Becomes a Book Becomes a Play

Sarah Ruhl’s Dear Elizabeth: A Play in Letters From Elizabeth Bishop to Robert Lowell and Back Again takes 900 pages of correspondence between the two poets and turns them into an on-stage performance.

September 26 • 7:00 AM

Sonic Hedgehog, DICER, and the Problem With Naming Genes

Wait, why is there a Pokemon gene?

September 26 • 6:00 AM

Sounds Like the Blues

At a music-licensing firm, any situation can become nostalgic, romantic, or adventurous, given the right background sounds.

September 26 • 5:00 AM

The Dark Side of Empathy

New research finds the much-lauded feeling of identification with another person’s emotions can lead to unwarranted aggressive behavior.

September 25 • 4:00 PM

Forging a New Path: Working to Build the Perfect Wildlife Corridor

When it comes to designing wildlife corridors, our most brilliant analytical minds are still no match for Mother Nature. But we’re getting there.

September 25 • 2:00 PM

Fashion as a Inescapable Institution

Like it or not, fashion is an institution because we can no longer feasibly make our own clothes.

September 25 • 12:00 PM

The Fake Birth Mothers Who Bilk Couples Out of Their Cash by Promising Future Babies

Another group that’s especially vulnerable to scams and fraud is that made up of those who are desperate to adopt a child.

September 25 • 10:03 AM

The Way We QuickType

September 25 • 10:00 AM

There’s a Name for Why You Feel Obligated to Upgrade All of Your Furniture to Match

And it’s called the Diderot effect.

Follow us

Trust Is Waning, and Inequality May Be To Blame

Trust in others and confidence in institutions is declining, while economic inequality creeps up, a new study shows.

Dopamine Might Be Behind Impulsive Behavior

A monkey study suggests the brain chemical makes what's new and different more attractive.

School Counselors Do More Than You’d Think

Adding just one counselor to a school has an enormous impact on discipline and test scores, according to a new study.

How a Second Language Trains Your Brain for Math

Second languages strengthen the brain's executive control circuits, with benefits beyond words.

Would You Rather Go Blind or Lose Your Mind?

Americans consistently fear blindness, but how they compare it to other ailments varies across racial lines.

The Big One

One company, Amazon, controls 67 percent of the e-book market in the United States—down from 90 percent five years ago. September/October 2014 new-big-one-5

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.