Menus Subscribe Search

Sociological Images

noah-film

Noah. (Photo: Paramount Pictures)

Why Survey Questions Matter: Blasphemy Edition

• March 19, 2014 • 4:00 PM

Noah. (Photo: Paramount Pictures)

“As a Faith Driven Consumer, are you satisfied with a Biblically themed movie — designed to appeal to you — which replaces the Bible’s core message with one created by Hollywood?”

“How could we get evidence for this?” I often ask students. And the answer, almost always is, “Do a survey.” The word survey has magical power; anything designated by that name wears a cloak of infallibility.

“Survey just means asking a bunch of people a bunch of questions,” I’ll say. “Whether it has any value depends on how good the bunch of people is and how good the questions are.” My hope is that a few examples of bad sampling and bad questions will demystify.

For example, Variety:

115

Here’s the lede:

Despite its Biblical inspiration, Paramount’s upcoming “Noah” may face some rough seas with religious audiences, according to a new survey by Faith Driven Consumers.

The data to confirm that idea:

The religious organization found in a survey that 98% of its supporters were not “satisfied” with Hollywood’s take on religious stories such as “Noah,” which focuses on Biblical figure Noah.

The sample:

Faith Driven Consumers surveyed its supporters over several days and based the results on a collected 5,000+ responses.

And (I’m saving the best for last) here’s the crucial survey question:

As a Faith Driven Consumer, are you satisfied with a Biblically themed movie — designed to appeal to you — which replaces the Bible’s core message with one created by Hollywood?

As if the part about “replacing the Bible’s core message” weren’t enough, the item reminds the respondent of her or his identity as a Faith Driven Consumer. It does make you wonder about that two percent who either were fine with the Hollywood* message or didn’t know.

You can’t really fault Faith Driven Consumer too much for this shoddy “research.” They’re not in business to find the sociological facts. What’s appalling is that Variety accepts it at face value and without comment.

*The director of Noah is Daniel Aronofsky; the script is credited to him and Ari Handel. For the Faith Driven Consumer, “Hollywood” may carry connotations in addition to that of industry and location—perhaps something similar to “New York sense of humor” in this clip  from The West Wing (the whole six minutes is worth watching, but you’ll get the idea if you push the pointer to 2:20 or so and watch for the next 45 seconds).


This post originally appeared on Sociological Images, a Pacific Standard partner site.

Jay Livingston
Jay Livingston is the chair of the Sociology Department at Montclair State University.

More From Jay Livingston

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

August 27 • 4:00 AM

The Politics of Anti-NIMBYism and Addressing Housing Affordability

Respected expert economists like Paul Krugman and Edward Glaeser are confusing readers with their poor grasp of demography.


August 26 • 4:00 PM

Marching in Sync May Increase Aggression

Another danger of militarizing the police: Marching in lock step doesn’t just intimidate opponents. It impacts the mindset of the marchers.


August 26 • 3:03 PM

The Best Reporting on the Federal Push to Militarize Local Police With Riot Gear, Armored Vehicles, and Assault Rifles

A few facts you might have missed about the flow of military equipment and tactics to local law enforcement.


August 26 • 2:00 PM

How the Other 23 Percent Live

Almost one-fourth of all children in the United States are now living in poverty, an increase of three million kids since 2005.


August 26 • 12:00 PM

Why Sports Need Randomness

Noah Davis talks to David Sally, one of the authors of The Numbers Game: Why Everything You Know About Soccer Is Wrong, about how uncertainty affects and enhances the games we watch.


August 26 • 10:00 AM

Honor: The Cause of—and Solution to—All of Society’s Problems

Recent research on honor culture, associated with the American South and characterized by the need to retaliate against any perceived improper conduct, goes way beyond conventional situations involving disputes and aggression.



August 26 • 8:00 AM

The Transformative Effects of Bearing Witness

How witnessing inmate executions affects those who watch, and how having an audience present can also affect capital punishment process and policy.



August 26 • 7:15 AM

Being a Couch Potato: Not So Bad After All?

For those who feel guilty about watching TV, a new study provides redemption.


August 26 • 6:00 AM

Redesigning Birth Control in the Developing World

How single-use injectable contraceptives could change family planning in Africa.


August 26 • 4:15 AM

How Gay Men Feel About Aging

Coming to terms with growing old can be difficult in the gay community. But middle-aged men are inventing new strategies to cope.


August 25 • 4:00 PM

What to Look for In Dueling Autopsies of Michael Brown

The postmortem by Michael Baden is only the beginning as teams of specialists study the body of an 18-year-old African American killed by police in Ferguson, Missouri.


August 25 • 2:00 PM

Thoughts That Can’t Be Thought and Ideas That Can’t Be Formed: The Promise of Smart Drugs

Are we asking the right questions about smart drugs? Marek Kohn looks at what they can do for us—and what they can’t.


August 25 • 12:00 PM

Does Randomness Actually Exist?

Our human minds are incapable of truly answering that question.


August 25 • 10:31 AM

Cesareans Are Still Best for Feet-First Babies

A new study confirms that surgery is the safest way to deliver a breech fetus.


August 25 • 10:00 AM

What Can Hurricanes Teach Us About Socioeconomic Mobility?

Hurricane Katrina wrought havoc on New Orleans but, nine years later, is there a silver lining to be found?


August 25 • 8:00 AM

How Low Voter Turnout Helps Public Employees

To a surprising degree, as voter turnout goes down, public employee compensation goes up.


August 25 • 6:00 AM

Beyoncé Isn’t an Anti-Feminist Terrorist

A new book called Staging the Blues shows she’s embracing a tradition of multi-dimensional stardom, rather than one racist trope.


August 25 • 4:00 AM

A Tale of Two Abortion Wars

While pro-life activists fight to rescue IVF embryos from the freezer, pregnant women in their third trimester with catastrophic fetal anomalies have nowhere to turn.


August 22 • 4:00 PM

The Invention of the Illegal Immigrant

It’s only fairly recently that we started to use the term that’s so popular right now.



August 22 • 2:00 PM

What Can U.S. Health Care Learn From the Ebola Outbreak?

A conversation with Jeanine Thomas, patient advocate, active member of ProPublica’s Patient Harm Facebook Community, and founder and president of the MRSA Survivors Network.


August 22 • 1:22 PM

Two Executions and the Unity of Mourning

The recent deaths of Michael Brown and James Foley, while worlds apart, are both emblematic of the necessity for all of us to fight to uphold the sanctity of human dignity and its enduring story.


August 22 • 10:00 AM

Turbo Paul: Art Thief Turned Art Crime Ombudsman

There’s art theft, there’s law enforcement, and, somewhere in between, there’s Turbo Paul.


Follow us


Subscribe Now

Being a Couch Potato: Not So Bad After All?

For those who feel guilty about watching TV, a new study provides redemption.

How Gay Men Feel About Aging

Coming to terms with growing old can be difficult in the gay community. But middle-aged men are inventing new strategies to cope.

Cesareans Are Still Best for Feet-First Babies

A new study confirms that surgery is the safest way to deliver a breech fetus.

The Impossibility of the Night Shift

Many night workers get “shift-work sleep disorder.” And no one knows how to treat it.

How the Brains of Risk-Taking Teens Work

There's heightened functional connectivity between the brain's emotion regulator and reason center, according to a recent neuroscience paper.

The Big One

One in two full-time American fast-food workers' families are enrolled in public assistance programs, at a cost of $7 billion per year. July/August 2014 fast-food-big-one

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.