Menus Subscribe Search

Follow us


Findings

research-papers

(Photo: Protasov AN/Shutterstock)

Linguistic Analysis Reveals Research Fraud

• August 28, 2014 • 4:00 AM

(Photo: Protasov AN/Shutterstock)

An examination of papers by the discredited Diederik Stapel finds linguistic differences between his legitimate and fraudulent studies.

Does the name Diederik Stapel ring a bell? He’s the prominent Dutch psychologist who, in 2011, was found to have engaged in research fraud on a massive scale. Much of its data, it now appears, was simply made up.

Could we—should we—have realized his too-good-to-be-true findings were, in fact, fiction? More importantly, can we spot the next guy whose provocative assertions are based on fraudulent data?

It’s a difficult task, to be sure. But David Markowitz and Jeffrey Hancock of Cornell University report that, in Stapel’s case, they were able to classify his research as legitimate or fraudulent “with above-chance accuracy” through careful linguistic analysis.

“Words such as ‘profoundly,’ ‘extremely’ and ‘considerably’ frame the (false) findings as having a substantial and dramatic impact.”

Looking at 49 papers he authored—24 fraudulent, 25 legitimate—they found telltale differences in his writing that indicated, at least to an extent, whether the research was real. They conclude that, even in “highly edited” scientific papers, “deception can be revealed.”

Using Wmatrix, a tool that provides linguistic analysis by investigating such variables as word frequency and grammar, Markowitz and Hancock found Stapel’s writing style varied in several ways when he described fake, rather than genuine, data.

Tellingly, “Stapel used nearly 3,000 fewer adjectives in his fake papers than his genuine papers,” they write in the online journal PLoS One. This pattern is consistent with the theory that “descriptive recalls of real experiences are more sensory and contextually driven.”

“Stapel also wrote with more certainty when describing his take data,” the researchers add, “using nearly one-third more certainty terms than he did in the genuine articles. Words such as ‘profoundly,’ ‘extremely’ and ‘considerably’ frame the (false) findings as having a substantial and dramatic impact.”

In other words, when the results were real, he didn’t feel the need to be quite so emphatic about their importance.

The number of experiments and references per paper did not differ significantly between the real and fake studies. However, the fraudulent papers had fewer authors, on average, than the genuine ones—no surprise, as it is “typically easier to deceive in the presence of a smaller group.”

Of course, other deceptive researchers may not leave precisely the same linguistic clues. But this study provides evidence that, even when dealing with the often-dry, heavily edited world of research papers, “language cues are important in deception detection.”

Markowitz and Hancock’s paper, by the way, is available free online. Feel free to count the number of adjectives they use.

Tom Jacobs
Staff writer Tom Jacobs is a veteran journalist with more than 20 years experience at daily newspapers. He has served as a staff writer for The Los Angeles Daily News and the Santa Barbara News-Press. His work has also appeared in The Los Angeles Times, Chicago Tribune, and Ventura County Star.

More From Tom Jacobs

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

December 20 • 10:28 AM

Flare-Ups

Are my emotions making me ill?


December 19 • 4:00 PM

How a Drug Policy Reform Organization Thinks of the Children

This valuable, newly updated resource for parents is based in the real world.


December 19 • 2:00 PM

Where Did the Ouija Board Come From?

It wasn’t just a toy.


December 19 • 12:00 PM

Social Scientists Can Do More to Eradicate Racial Oppression

Using our knowledge of social systems, all social scientists—black or white, race scholar or not—have an opportunity to challenge white privilege.


December 19 • 10:17 AM

How Scientists Contribute to Bad Science Reporting

By not taking university press officers and research press releases seriously, scientists are often complicit in the media falsehoods they so often deride.


December 19 • 10:00 AM

Pentecostalism in West Africa: A Boon or Barrier to Disease?

How has Ghana stayed Ebola-free despite being at high risk for infection? A look at their American-style Pentecostalism, a religion that threatens to do more harm than good.


December 19 • 8:00 AM

Don’t Text and Drive—Especially If You’re Old

A new study shows that texting while driving becomes even more dangerous with age.


December 19 • 6:12 AM

All That ‘Call of Duty’ With Your Friends Has Not Made You a More Violent Person

But all that solo Call of Duty has.


December 19 • 4:00 AM

Food for Thought: WIC Works

New research finds participation in the federal WIC program, which subsidizes healthy foods for young children, is linked with stronger cognitive development and higher test scores.


December 18 • 4:00 PM

How I Navigated Life as a Newly Sober Mom

Saying “no” to my kids was harder than saying “no” to alcohol. But for their sake and mine, I had to learn to put myself first sometimes.


December 18 • 2:00 PM

Women in Apocalyptic Fiction Shaving Their Armpits

Because our interest in realism apparently only goes so far.


December 18 • 12:00 PM

The Paradox of Choice, 10 Years Later

Paul Hiebert talks to psychologist Barry Schwartz about how modern trends—social media, FOMO, customer review sites—fit in with arguments he made a decade ago in his highly influential book, The Paradox of Choice: Why More Is Less.


December 18 • 10:00 AM

What It’s Like to Spend a Few Hours in the Church of Scientology

Wrestling with thetans, attempting to unlock a memory bank, and a personality test seemingly aimed at people with depression. This is Scientology’s “dissemination drill” for potential new members.


December 18 • 8:00 AM

Gendering #BlackLivesMatter: A Feminist Perspective

Black men are stereotyped as violent, while black women are rendered invisible. Here’s why the gendering of black lives matters.


December 18 • 7:06 AM

Apparently You Can Bring Your Religion to Work

New research says offices that encourage talk of religion actually make for happier workplaces.


December 18 • 6:00 AM

The Very Weak and Complicated Links Between Mental Illness and Gun Violence

Vanderbilt University’s Jonathan Metzl and Kenneth MacLeish address our anxieties and correct our assumptions.


December 18 • 4:00 AM

Should Movies Be Rated RD for Reckless Driving?

A new study finds a link between watching films featuring reckless driving and engaging in similar behavior years later.


December 17 • 4:00 PM

How to Run a Drug Dealing Network in Prison

People tend not to hear about the prison drug dealing operations that succeed. Substance.com asks a veteran of the game to explain his system.


December 17 • 2:00 PM

Gender Segregation of Toys Is on the Rise

Charting the use of “toys for boys” and “toys for girls” in American English.


December 17 • 12:41 PM

Why the College Football Playoff Is Terrible But Better Than Before

The sample size is still embarrassingly small, but at least there’s less room for the availability cascade.


December 17 • 11:06 AM

Canadian Kids Have a Serious Smoking Problem

Bootleg cigarette sales could be leading Canadian teens to more serious drugs, a recent study finds.


December 17 • 10:37 AM

A Public Lynching in Sproul Plaza

When photographs of lynching victims showed up on a hallowed site of democracy in action, a provocation was issued—but to whom, by whom, and why?


December 17 • 8:00 AM

What Was the Job?

This was the year the job broke, the year we accepted a re-interpretation of its fundamental bargain and bought in to the push to get us to all work for ourselves rather than each other.


December 17 • 6:00 AM

White Kids Will Be Kids

Even the “good” kids—bound for college, upwardly mobile—sometimes break the law. The difference? They don’t have much to fear. A professor of race and social movements reflects on her teenage years and faces some uncomfortable realities.



Follow us


Don’t Text and Drive—Especially If You’re Old

A new study shows that texting while driving becomes even more dangerous with age.

Apparently You Can Bring Your Religion to Work

New research says offices that encourage talk of religion actually make for happier workplaces.

Canadian Kids Have a Serious Smoking Problem

Bootleg cigarette sales could be leading Canadian teens to more serious drugs, a recent study finds.

The Hidden Psychology of the Home Ref

That old myth of home field bias isn’t a myth at all; it’s a statistical fact.

The Big One

One in two United States senators and two in five House members who left office between 1998 and 2004 became lobbyists. November/December 2014

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.