Menus Subscribe Search

Follow us


digital-polling

(ILLUSTRATION: PAVEL IGNATOV/SHUTTERSTOCK)

The Use and Abuse of Polling in American Politics

• September 16, 2013 • 8:00 AM

(ILLUSTRATION: PAVEL IGNATOV/SHUTTERSTOCK)

Last week’s fight over PPP’s decision to hold back the results of a poll highlight how too many pollsters operating in the political sphere take an Ivory Tower attitude, disavowing responsibility for the consequences of their work.

An epic data nerd battle started last Wednesday over Twitter as critics lambasted Public Policy Polling (PPP) for withholding a weekend poll of voters in Pueblo County, Colorado.

The poll asked Pueblo County voters if they supported the legislative recall of State Senator Angela Giron, a moderate Democrat who supported stronger gun control legislation in her state. Giron’s Tuesday recall election and that of State Senate President John Morse, of Colorado Springs, marked the first legislative recalls of state lawmakers in Colorado’s 137-year history.

PPP found 54 percent of Pueblo’s registered voters overall favored Giron’s recall, while just 42 percent opposed it. Stranger yet, a whopping 33 percent of Democratic respondents said that they supported the recall. Consider that 47 percent of Pueblo’s registered voters are Democrats; just 23 percent are Republicans.

“In a district that Barack Obama won by almost 20 points I figured there was no way that could be right,” a post from PPP Director Tom Jensen explained on the firm’s website. Not only that, voters in Pueblo County support expanded background checks for gun buyers—68 to 27 percent—and are split on ammunition limits.

“VERY bad and unscientific practice for @ppppolls to suppress a polling result they didn’t believe/didn’t like.”

PPP suspected a design flaw. “This was the first legislative recall election in Colorado history. There’s been a lot of voter confusion,” Jensen noted. “That finding made me think that respondents may not have understood what they were being asked.” With no time to test a new survey design, they waited for the election results.

Tuesday’s election results matched the unpublished poll results closely: 56 percent for the recall, 44 percent against. On Wednesday morning, PPP posted the weekend poll with an explanation for the delay and analysis. Poll trolling commenced.

PPP is not a public agency, it’s not a non-profit, and it’s not under any obligation to anyone but its clients to release its polls. The group has nevertheless made a policy of being responsive to public interest and shown a commitment to transparency. That’s laudable, not lamentable.

But Nate Silver couldn’t help himself: the much-feted statistician behind FiveThirtyEight threw a punch at the pollsters over Twitter early Wednesday afternoon, announcing, “VERY bad and unscientific practice for @ppppolls to suppress a polling result they didn’t believe/didn’t like.” He linked to the PPP post explaining their delay in releasing the Giron poll results. In his next-day follow-up, Silver accused PPP of an “approach to polling [that] is extremely ad hoc.” He added, “that ad-hockery stems from a lack of appreciation/understanding for the statistical fundamentals behind polling.”

Taking the cue from golden-boy Silver, other commentators and analysts have piled on. Mark Blumenthal, founding editor of Pollster.com and senior polling editor for the Huffington Post, rallied behind Silver by early afternoon Wednesday. He launched into his criticisms of PPP with a quick disclaimer: “I’m late to this #NerdFight.” Thursday brought a particularly pointed piece by Nate Cohn, The New Republic’s self-professed amateur polling analyst. Blumenthal rehashed Cohn’s arguments in a post for the Huffington Post.

It’s worth noting that, with the exception of Blumenthal, few of PPP’s critics have polling chops; journalists and practitioners with experience rallied behind PPP.

Silver criticizing a polling outfit for being reckless and unscientific is like Lindsay Lohan accusing Meryl Streep of lacking work ethic. In November 2011, Silver declared that President Obama had just a 17 percent chance of re-election. I wrote about the many problems with Silver’s piece, statistical and ethical, for the Huffington Post: Silver conceded he used just 17 elections (the post-1944 presidential cycles), discounted the reliability of historical precedent, admitted the inconsistency of economic forecasts, and exposed prevailing methods for predicting electoral outcome through tables and rundowns as no better than hit-or-miss.

In one corner, we have a hitherto respected polling outfit that releases data regularly; in the other, a statistician who strung together a series of untrustworthy variables to land the cover of the New York Times Magazine, justifying the exercise by listing the many problems with his claim in the body of his piece.

Gawker’s Max Read noted, “nate silver is a funny guy to be knocking an organization for lack of transparency!” The Plum Line’s Greg Sargent asked Nate Silver, “are you alleging they ‘suppressed’ the result because they ‘didn’t like’ it?” That “[PPP-Nate Silver] dispute turns out to be pretty narrow,” Brian Beutler of Salon commented. I found myself drawn into the Twitter war on the topic: “If you mixed blue and yellow and got purple, would you announce it before re-examining your experiment?”

It’s refreshing to find that there are still pollsters who self-regulate. Too many pollsters operating in the political sphere take an Ivory Tower attitude, disavowing responsibility for the consequences of their polls and analyses. Their reasoning, as I understand it, is that, so long as they note the margin of error and any other qualifications in fine print, they’re on the up-and-up. On that, they’re wrong.

Pre-election polling can affect election results—even Nate Silver admits as much. We’ve got multiple theories to choose from, each of which carries some weight. Voters may want to choose the winner or vote with their friends, for example. Low-information voters may take a marketplace approach, relying on public opinion to champion the worthiest candidate.

When a pollster puts out a poll with a huge margin of error or a skewed sample frame, most coverage won’t note the fine print, even if the pollster does. The irresponsible pollster publishes the poll, content to slap on a sensational headline or else criticize media for doing so. The responsible pollster recognizes the dangers and acts to ensure that data isn’t misappropriated.

PPP identified a problematic result and held it until they could explain what happened, despite the fact that they could’ve gotten publicity from the polling before the election. They published the poll after the election knowing they might face sustained criticism for having held it back. That’s a commitment to science and public responsibility that we should welcome.

Rebecca Buckwalter-Poza
Rebecca Buckwalter-Poza attended Harvard College and Yale Law School. She has written on law and politics for the Nation, the Atlantic, Politico, the Daily Beast, and CNN, and co-authored James Carville’s 40 More Years. Follow her on Twitter @rpbp.

More From Rebecca Buckwalter-Poza

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

October 1 • 1:00 PM

Midlife Neuroticism Linked to Alzheimer’s Disease in Old Age

New research from Sweden suggests that the personality dimension is connected to who ultimately suffers from late-in-life dementia.



October 1 • 11:11 AM

The Creative Class Boondoggle in Downtown Las Vegas

On Tony Hsieh and the pseudoscience of “collisions.”


October 1 • 9:14 AM

Mysterious Resting State Networks Might Be What Allow Different Brain Therapies to Work

Deep brain stimulation and similar treatments target the hubs of larger resting-state networks in the brain, researchers find.


October 1 • 6:00 AM

Would You Like a Subscription with Your Coffee?

A new app hopes to unite local coffee shops while helping you find a cheap cup of good coffee.


October 1 • 4:00 AM

How to Plant a Library

Somewhere outside of Oslo, there are 1,000 newly-planted spruce trees. One hundred years from now, if everything goes to plan, they’ll be published together as 100 pieces of art.



September 30 • 10:09 AM

Trust Is Waning, and Inequality May Be to Blame

Trust in others and confidence in institutions is declining, while economic inequality creeps up, a new study shows.


September 30 • 8:00 AM

The Psychology of Penmanship

Graphology: It’s all (probably) bunk.



September 30 • 6:00 AM

The Medium Is the Message, 50 Years Later

Five decades on, what can Marshall McLuhan’s Understanding Media tell us about today?


September 30 • 4:00 AM

Grad School’s Mental Health Problem

Navigating the emotional stress of doctoral programs in a down market.


September 29 • 1:21 PM

Conference Call: Free Will Conference


September 29 • 12:00 PM

How Copyright Law Protects Art From Criticism

A case for allowing the copyright on Gone With the Wind to expire.


September 29 • 10:00 AM

Should We Be Told Who Funds Political Attack Ads?

On the value of campaign finance disclosure.


September 29 • 8:00 AM

Searching for a Man Named Penis

A quest to track down a real Penis proves difficult.


September 29 • 6:00 AM

Why Do So Many People Watch HGTV?

The same reason so many people watch NCIS or Law and Order: It’s all a procedural.


September 29 • 4:00 AM

The Link Between Depression and Terrorism

A new study from the United Kingdom finds a connection between depression and radicalization.


September 26 • 4:00 PM

Fast Track to a Spill?

Oil pipeline projects across America are speeding forward without environmental review.


September 26 • 2:00 PM

Why Liberals Love the Disease Theory of Addiction, by a Liberal Who Hates It

The disease model is convenient to liberals because it spares them having to say negative things about poor communities. But this conception of addiction harms the very people we wish to help.


September 26 • 1:21 PM

Race, Trust, and Split-Second Judgments


September 26 • 9:47 AM

Dopamine Might Be Behind Impulsive Behavior

A monkey study suggests the brain chemical makes what’s new and different more attractive.


September 26 • 8:00 AM

A Letter Becomes a Book Becomes a Play

Sarah Ruhl’s Dear Elizabeth: A Play in Letters From Elizabeth Bishop to Robert Lowell and Back Again takes 900 pages of correspondence between the two poets and turns them into an on-stage performance.


September 26 • 7:00 AM

Sonic Hedgehog, DICER, and the Problem With Naming Genes

Wait, why is there a Pokemon gene?


September 26 • 6:00 AM

Sounds Like the Blues

At a music-licensing firm, any situation can become nostalgic, romantic, or adventurous, given the right background sounds.


Follow us


Mysterious Resting State Networks Might Be What Allow Different Brain Therapies to Work

Deep brain stimulation and similar treatments target the hubs of larger resting-state networks in the brain, researchers find.

Trust Is Waning, and Inequality May Be to Blame

Trust in others and confidence in institutions is declining, while economic inequality creeps up, a new study shows.

Dopamine Might Be Behind Impulsive Behavior

A monkey study suggests the brain chemical makes what's new and different more attractive.

School Counselors Do More Than You’d Think

Adding just one counselor to a school has an enormous impact on discipline and test scores, according to a new study.

How a Second Language Trains Your Brain for Math

Second languages strengthen the brain's executive control circuits, with benefits beyond words.

The Big One

One company, Amazon, controls 67 percent of the e-book market in the United States—down from 90 percent five years ago. September/October 2014 new-big-one-5

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.