Menus Subscribe Search

Follow us


Sociological Images

noah-film

Noah. (Photo: Paramount Pictures)

Why Survey Questions Matter: Blasphemy Edition

• March 19, 2014 • 4:00 PM

Noah. (Photo: Paramount Pictures)

“As a Faith Driven Consumer, are you satisfied with a Biblically themed movie — designed to appeal to you — which replaces the Bible’s core message with one created by Hollywood?”

“How could we get evidence for this?” I often ask students. And the answer, almost always is, “Do a survey.” The word survey has magical power; anything designated by that name wears a cloak of infallibility.

“Survey just means asking a bunch of people a bunch of questions,” I’ll say. “Whether it has any value depends on how good the bunch of people is and how good the questions are.” My hope is that a few examples of bad sampling and bad questions will demystify.

For example, Variety:

115

Here’s the lede:

Despite its Biblical inspiration, Paramount’s upcoming “Noah” may face some rough seas with religious audiences, according to a new survey by Faith Driven Consumers.

The data to confirm that idea:

The religious organization found in a survey that 98% of its supporters were not “satisfied” with Hollywood’s take on religious stories such as “Noah,” which focuses on Biblical figure Noah.

The sample:

Faith Driven Consumers surveyed its supporters over several days and based the results on a collected 5,000+ responses.

And (I’m saving the best for last) here’s the crucial survey question:

As a Faith Driven Consumer, are you satisfied with a Biblically themed movie — designed to appeal to you — which replaces the Bible’s core message with one created by Hollywood?

As if the part about “replacing the Bible’s core message” weren’t enough, the item reminds the respondent of her or his identity as a Faith Driven Consumer. It does make you wonder about that two percent who either were fine with the Hollywood* message or didn’t know.

You can’t really fault Faith Driven Consumer too much for this shoddy “research.” They’re not in business to find the sociological facts. What’s appalling is that Variety accepts it at face value and without comment.

*The director of Noah is Daniel Aronofsky; the script is credited to him and Ari Handel. For the Faith Driven Consumer, “Hollywood” may carry connotations in addition to that of industry and location—perhaps something similar to “New York sense of humor” in this clip  from The West Wing (the whole six minutes is worth watching, but you’ll get the idea if you push the pointer to 2:20 or so and watch for the next 45 seconds).


This post originally appeared on Sociological Images, a Pacific Standard partner site.

Jay Livingston
Jay Livingston is the chair of the Sociology Department at Montclair State University.

More From Jay Livingston

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

October 24 • 4:00 PM

We Need to Normalize Drug Use in Our Society

After the disastrous misconceptions of the 20th century, we’re returning to the idea that drugs are an ordinary part of life experience and no more cause addiction than do other behaviors. This is rational and welcome.


October 24 • 2:00 PM

A Letter to the Next Attorney General: Fix Presidential Pardons

More than two years ago, a series showed that white applicants were far more likely to receive clemency than comparable applicants who were black. Since then, the government has spent hundreds of thousands of dollars on a study, but the pardons system remains unchanged.


October 24 • 12:00 PM

What Makes You So Smart, Middle School Math Teacher?

Noah Davis talks to Vern Williams about what makes middle school—yes, middle school—so great.


October 24 • 10:00 AM

Why DNA Is One of Humanity’s Greatest Inventions

How we’ve co-opted our genetic material to change our world.


October 24 • 8:00 AM

What Do Clowns Think of Clowns?

Three major players weigh in on the current state of the clown.


October 24 • 7:13 AM

There Is No Surge in Illegal Immigration

The overall rate of illegal immigration has actually decreased significantly in the last 10 years. The time is ripe for immigration reform.


October 24 • 6:15 AM

Politicians Really Aren’t Better Decision Makers

Politicians took part in a classic choice experiment but failed to do better than the rest of us.


October 24 • 5:00 AM

Why We Gossip: It’s Really All About Ourselves

New research from the Netherlands finds stories we hear about others help us determine how we’re doing.


October 24 • 2:00 AM

Congratulations, Your City Is Dying!

Don’t take population numbers at face value.


October 23 • 4:00 PM

Of Course Marijuana Addiction Exists

The polarized legalization debate leads to exaggerated claims and denials about pot’s potential harms. The truth lies somewhere in between.


October 23 • 2:00 PM

American Companies Are Getting Way Too Cozy With the National Security Agency

Newly released documents describe “contractual relationships” between the NSA and U.S. companies, as well as undercover operatives.


October 23 • 12:00 PM

The Man Who’s Quantifying New York City

Noah Davis talks to the proprietor of I Quant NY. His methodology: a little something called “addition.”


October 23 • 11:02 AM

Earliest High-Altitude Settlements Found in Peru

Discovery suggests humans adapted to high altitude faster than previously thought.


October 23 • 10:00 AM

The Psychology of Bribery and Corruption

An FBI agent offered up confidential information about a political operative’s enemy in exchange for cash—and they both got caught. What were they thinking?


October 23 • 8:00 AM

Ebola News Gives Me a Guilty Thrill. Am I Crazy?

What it means to feel a little excited about the prospect of a horrific event.


October 23 • 7:04 AM

Why Don’t Men Read Romance Novels?

A lot of men just don’t read fiction, and if they do, structural misogyny drives them away from the genre.


October 23 • 6:00 AM

Why Do Americans Pray?

It depends on how you ask.


October 23 • 4:00 AM

Musicians Are Better Multitaskers

New research from Canada finds trained musicians more efficiently switch from one mental task to another.


October 22 • 4:00 PM

The Last Thing the Women’s Movement Needs Is a Heroic Male Takeover

Is the United Nations’ #HeForShe campaign helping feminism?


October 22 • 2:00 PM

Turning Public Education Into Private Profits

Baker Mitchell is a politically connected North Carolina businessman who celebrates the power of the free market. Every year, millions of public education dollars flow through Mitchell’s chain of four non-profit charter schools to for-profit companies he controls.


October 22 • 12:00 PM

Will the End of a Tax Loophole Kill Off Irish Business and Force Google and Apple to Pay Up?

U.S. technology giants have constructed international offices in Dublin in order to take advantage of favorable tax policies that are now changing. But Ireland might have enough other draws to keep them there even when costs climb.


October 22 • 10:00 AM

Veterans in the Ivory Tower

Why there aren’t enough veterans at America’s top schools—and what some people are trying to do to change that.


October 22 • 8:00 AM

Our Language Prejudices Don’t Make No Sense

We should embrace the fact that there’s no single recipe for English. Making fun of people for replacing “ask” with “aks,” or for frequently using double negatives just makes you look like the unsophisticated one.


October 22 • 7:04 AM

My Politicians Are Better Looking Than Yours

A new study finds we judge the cover by the book—or at least the party.


October 22 • 6:00 AM

How We Form Our Routines

Whether it’s a morning cup of coffee or a glass of warm milk before bed, we all have our habitual processions. The way they become engrained, though, varies from person to person.


Follow us


Politicians Really Aren’t Better Decision Makers

Politicians took part in a classic choice experiment but failed to do better than the rest of us.

Earliest High-Altitude Settlements Found in Peru

Discovery suggests humans adapted to high altitude faster than previously thought.

My Politicians Are Better Looking Than Yours

A new study finds we judge the cover by the book—or at least the party.

That Cigarette Would Make a Great Water Filter

Clean out the ashtray, add some aluminum oxide, and you've (almost) got yourself a low-cost way to remove arsenic from drinking water.

Love and Hate in Israel and Palestine

Psychologists find that parties to a conflict think they're motivated by love while their enemies are motivated by hate.

The Big One

One company, Amazon, controls 67 percent of the e-book market in the United States—down from 90 percent five years ago. September/October 2014 new-big-one-5

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.