Menus Subscribe Search

Sociological Images

noah-film

Noah. (Photo: Paramount Pictures)

Why Survey Questions Matter: Blasphemy Edition

• March 19, 2014 • 4:00 PM

Noah. (Photo: Paramount Pictures)

“As a Faith Driven Consumer, are you satisfied with a Biblically themed movie — designed to appeal to you — which replaces the Bible’s core message with one created by Hollywood?”

“How could we get evidence for this?” I often ask students. And the answer, almost always is, “Do a survey.” The word survey has magical power; anything designated by that name wears a cloak of infallibility.

“Survey just means asking a bunch of people a bunch of questions,” I’ll say. “Whether it has any value depends on how good the bunch of people is and how good the questions are.” My hope is that a few examples of bad sampling and bad questions will demystify.

For example, Variety:

115

Here’s the lede:

Despite its Biblical inspiration, Paramount’s upcoming “Noah” may face some rough seas with religious audiences, according to a new survey by Faith Driven Consumers.

The data to confirm that idea:

The religious organization found in a survey that 98% of its supporters were not “satisfied” with Hollywood’s take on religious stories such as “Noah,” which focuses on Biblical figure Noah.

The sample:

Faith Driven Consumers surveyed its supporters over several days and based the results on a collected 5,000+ responses.

And (I’m saving the best for last) here’s the crucial survey question:

As a Faith Driven Consumer, are you satisfied with a Biblically themed movie — designed to appeal to you — which replaces the Bible’s core message with one created by Hollywood?

As if the part about “replacing the Bible’s core message” weren’t enough, the item reminds the respondent of her or his identity as a Faith Driven Consumer. It does make you wonder about that two percent who either were fine with the Hollywood* message or didn’t know.

You can’t really fault Faith Driven Consumer too much for this shoddy “research.” They’re not in business to find the sociological facts. What’s appalling is that Variety accepts it at face value and without comment.

*The director of Noah is Daniel Aronofsky; the script is credited to him and Ari Handel. For the Faith Driven Consumer, “Hollywood” may carry connotations in addition to that of industry and location—perhaps something similar to “New York sense of humor” in this clip  from The West Wing (the whole six minutes is worth watching, but you’ll get the idea if you push the pointer to 2:20 or so and watch for the next 45 seconds).


This post originally appeared on Sociological Images, a Pacific Standard partner site.

Jay Livingston
Jay Livingston is the chair of the Sociology Department at Montclair State University.

More From Jay Livingston

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

September 15 • 2:04 AM

No Innovation Without Migration: Do Places Make People?

We know that people make places, but does it also work the other way?


September 12 • 4:00 PM

Life, Liberty, and the Pursuit of Plastic Bags

California wants you to pay for your plastic bags. (FYI: That’s not an infringement on your constitutional rights.)


September 12 • 2:00 PM

Should We Trust the Hearts of White People?

On the 50th anniversary of the Civil Rights Act, revisiting a clip of James Baldwin on the Dick Cavett Show.


September 12 • 12:00 PM

Big Government, Happy Citizens?

You may like to talk about how much happier you’d be if the government didn’t interfere with your life, but that’s not what the research shows.


September 12 • 10:00 AM

Whispering in the Town Square: Can Twitter Provide an Escape From All Its Noise?

Twitter has created its own buzzing, digital agora, but when users want to speak amongst themselves, they tend to leave for another platform. It’s a social network that helps you find people to talk to—but barely lets you do any talking.


September 12 • 9:03 AM

How Ancient DNA Is Rewriting Human History

We thought we knew how we’d been shaped by evolution. We were wrong.


September 12 • 8:02 AM

Give Yourself a Present for the Future

Psychologists discover that we underestimate the value of looking back.


September 12 • 8:00 AM

I Walked Through the Financial Crisis

Why are former Wall Street employees guiding tourists around the Financial District? Paul Hiebert signed himself up and tried to find out.


September 12 • 7:05 AM

Scams, Scams, Everywhere


September 12 • 6:17 AM

In Soccer as in Art, Motifs Matter

A new study suggests a way to quantitatively measure a team’s style through its pass flow. It may become another metric used to evaluate potential recruits.


September 12 • 4:00 AM

Comfort Food Is a Myth

New research finds that, contrary to our beliefs, such foods don’t have any special ability to improve our moods.



September 11 • 4:00 PM

Reading the Camouflage Uniforms in Ferguson: ‘You Are Now Enemy Combatants’

Why are police officers wearing green or desert camouflage in a suburban environment?


September 11 • 2:00 PM

Wage Theft: How Two States Are Fighting Against Companies That Categorize Employees as Independent Contractors

New York and Illinois have passed hard-nosed laws and taken an aggressive tack toward misclassification.


September 11 • 11:03 AM

Yes, I’m a Good Person. But Did You Hear About Her?

A new study tracks how people experience moral issues in everyday life.


September 11 • 11:00 AM

Searching for Everyday Morality

Experimenters use text messages to study morality beyond the lab.


September 11 • 8:00 AM

The Geography of Uber

If it continues to grow—and there are few reasons to think it won’t—will Uber transform the infrastructure of cities or glom onto what’s already there?



September 11 • 6:05 AM

One Man’s Search for an Orgasmic Life Force

It remains unclear what “orgone” actually is, but Wilhelm Reich thought you could find it by sitting inside a box.


September 11 • 4:03 AM

Jack the Ripper’s DNA: Was Aaron Kosminski Behind the Whitechapel Murders?

Russell Edwards says he’s solved the mystery. His proof might be a little threadbare.



September 10 • 4:00 PM

The Average White American’s Social Network Is Just One Percent Black

And three-quarters of white Americans report that they haven’t had a meaningful conversation with a single non-white person in the last six months.


September 10 • 2:00 PM

Eye on the Fly

The tiny fruit fly has been beloved by developmental biologists for more than a century. Turing patterns may yet explain its shape.


September 10 • 10:02 AM

Why Do Women Earn Less as Mothers and Men Earn More as Fathers?

For women, becoming a parent means you can expect to earn even less over your lifetime—unless you’re Marissa Mayer.



Follow us


Big Government, Happy Citizens?

You may like to talk about how much happier you'd be if the government didn't interfere with your life, but that's not what the research shows.

Give Yourself a Present for the Future

Psychologists discover that we underestimate the value of looking back.

In Soccer as in Art, Motifs Matter

A new study suggests a way to quantitatively measure a team’s style through its pass flow. It may become another metric used to evaluate potential recruits.

Searching for Everyday Morality

Experimenters use text messages to study morality beyond the lab.

Is Back Pain Ruining Your Sex Life?

You might be doing it wrong.

The Big One

One in three drivers in Brooklyn's Park Slope—at certain times of day—is just looking for parking. The same goes for drivers in Manhattan's SoHo. September/October 2014 new-big-one-3

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.