Questioning Questions in Climate Flip-Flops
The mass of Americans still accept reality of climate change, but a glut of complex polls manages to make that difficult to discern.
Dipping poll numbers among Americans concerned about climate change has some questioning the questions in the polls. (KREMA kreativni marketing / istockphoto)
March 12, 2010 • By • 7 Comments
It is a uniquely convoluted aspect of the climate change debate in Washington that any legislative solution won’t be grounded directly on science. Rather, policy will result from politicians’ perceptions of the public’s perception of science — a multistep chain that weakens the voice of natural scientists while boosting the power of a less-than-perfect scientific tool, the public opinion poll.
Jon Krosnick, a professor of communications, political science and psychology at Stanford University, has been mulling the curious recent case of dipping poll numbers among Americans concerned about climate change. The Pew Research Center prompted alarming headlines in October when it reported only 57 percent of Americans think there is solid evidence the world is warming, down from 71 percent just a year earlier. Similar trends have been trumpeted by numerous other surveys including, just Thursday, a new Gallup poll.
Krosnick, who offered the results of some of his own research in a Capitol Hill briefing today, gives the common caution of academics who study polling: Question wording matters.
Consider, first, that polls produce the most accurate and telling answers when they’re not heavily loaded with jargon, when they’re well balanced and when they ask a single question at a time.
Then consider these questions:
From the Pew poll: “From what you’ve read and heard, is there solid evidence that the average temperature on Earth has been getting warmer over the past few decades or not?”
THE IDEA LOBBY
Miller-McCune's Washington correspondent Emily Badger follows the ideas informing, explaining and influencing government, from the local think tank circuit to academic research that shapes D.C. policy from afar.
From a Yale survey: “Recently, you may have noticed that global warming has been getting some attention in the news. Global warming refers to the idea that the world’s average temperature has been increasing over the past 150 years, may be increasing more in the future, and that the world’s climate may change as a result. What do you think? Do you think that global warming is happening?”
From a CNN poll: “Which of the following statements comes closest to your view of global warming: Global warming is a proven fact and is mostly caused by emissions from cars and industrial facilities such as power plants and factories; global warming is a proven fact and is mostly caused by natural changes that have nothing to do with emissions from cars and industrial facilities; or, global warming is a theory that has not yet been proven.”
“A classic case of what we call a multi-barreled question — way too many things going on,” Krosnick said of the last inquiry. “Imagine hearing that in a telephone interview and trying to pick one of those options.”
Many climate questions place what Krosnick calls a “cognitive burden” on the answerer. Even the original Pew poll confusingly asks what respondents think about what they’ve “read and heard” about climate change, not what they believe about climate change themselves.
The timing of the polls also matters: Most organizations ask about climate change in the spring or summer around Earth Day. This latest round of anomalous polls, however, were conducted on the eve of the climate summit in Copenhagen in December, at a time when respondents were influenced, among other factors, by the weather outside when the phone rang.
In designing his own poll, Krosnick asks the question this way: “You may have heard about the idea that the world’s temperature may have been going up slowly over the past 100 years. What is your personal opinion about this? Do you think this has probably been happening, or do you think it probably has not been happening?”
In 1997, 79 percent of people went with the first option. By 2006, that number was up to the mid-80s. Today, it’s back at 80 percent. The most recent decrease is statistically significant, but as Krosnick stresses, 80 percent is still a remarkably high level of consensus among the American population.
So why the recent dip? Krosnick has eliminated four theories: that trust in scientists is on the wane; that the sour economy has dragged down our willingness to tackle (and recognize) the expensive problem; that skeptical Republican politicians have been getting more traction with the public; or that skeptical scientists have been getting more traction.
He settles instead on this fact: 2008 was an unusually cool year that broke the steady warming trend of the past decade. Krosnick divides us into two groups — people who trust natural scientists, and those who don’t. The second group, if not relying on scientific consensus, is more likely to answer the question drawing on observable experience like the temperature outside.
The recent drop in poll numbers, in short, is the result of a large change in opinion among a small group of people, more specifically people not inclined to listen to scientists anyway.
“If the natural scientists are right that the Earth’s temperatures will continue to rise and 2008 was an aberration,” Krosnick said, “as soon as those folks put their finger out the window and realize that 2010, 2011 are back on that track, they will flip back to the other side.”
Of course, the people who need to understand this are not the folks with their fingers out the window, but the politicians who misread their outsized influence on polling numbers to suggest the public at large has lost interest in climate change.
Or, as Krosnick suggests, we need to start polling the scientists themselves (and widely publicizing the results). Today, two-thirds of the population does not realize the scientific community has reached consensus on climate change. For many scientists, that discrepancy is a much greater mystery than the recent dip in poll numbers.
Sign up for our free e-newsletter.
Are you on Facebook? Become our fan.
Follow us on Twitter.
-
Neil Craig
-
Anonymous
-
Steven Douglas
-
pojoe
-
Meme Mine
-
emilybadger
-
Manfred Zysk, M.E.
