Menus Subscribe Search

Predicting House Races by Weight of Tweet

• October 31, 2012 • 4:00 AM

With the presidential prediction game dreadfully same-y, surely there’s another constantly changing fix for political junkies. How about forecasting all 435 U.S. congressional races every day, based on brand new data every day?

That’s what you get at “Voting With Your Tweet,” an experiment that mines mentions of congressional candidates in the Twitter-sphere to predict who will win each race and what the actual vote share will be. Unlike past efforts at using social media to predict political contests, which made their “predictions” after the voters had settled the matter, this research is happening now and you can check in at the California News Service site to see what’s up today, yesterday, or tomorrow.

Using social media as a forecasting tool is a hot area right now, and mixing Twitter with punditry is not brand new. Carnegie Mellon’s Brendan O’Connor compared tweeting to polling results in 2010, and not surprisingly found some correlation. Andranik Tumasjan and his colleagues at the Munich Technical University found that “the mere number of messages mentioning a party reflects the election result” in a German federal election. The “media utility” Tweetminster, meanwhile, says it accurately predicted the outcome of the last British Parliamentary elections using Twitter (an experiment they note is based on an earlier Japanese study looking at “online buzz and election results”).

Voting With Your Tweet is a kind of combination, using past results to predict future returns (but with mountains of caveats).

Analyzing about a quarter million tweets from 356 races in the 2010 mid-terms, UC Berkeley doctoral candidate Mark Huberty correctly “predicted” (after the fact) the winners 92 percent of the time, a much better showing than the six professional human pundits used for comparison. (The technical report on 2010 is available here, but be prepared to see that the 92 percent figure is one of a number of outcomes based on different statistical methods and statistical approaches. Huberty goes with “better than 85 percent accuracy” in the paper’s introduction.)

Taking that information, Huberty “trained two machine learning algorithms to determine what word features of those tweets best predicted whether the Democratic or Republican party candidate won each race.” So in its current incarnation …

We think the finished algorithm works like this:

First, it identifies from the language in a candidate’s tweets whether they are the incumbent or challenger. Since incumbents win about 85% of the time, this provides a good baseline.

It then adjusts the baseline prediction based on sentiment and action-related phrases. For instance, “voted hcr” (indicating that the incumbent voted for health care reform) was one of the most influential predictors alongside incumbency-related phrases. The algorithm weights those phrases positively or negatively, depending on how predictive they were of a candidate winning or losing.

And you can take this forecast to the bank? Only if you’re Lehman Brothers. “Voting with your Tweet is an experiment and should be treated as such,” Huberty and “data guru” Len DeGroot from the Knight Digital Media Center at Berkeley’s Graduate School of Journalism write in an honest and detailed set of FAQs. “We think this might work and think we might know why. But we could fail spectacularly.”

So why do it at all, or at least so publicly? The answers are refreshing:

Anyone who is interested can observe the experiment.
It keeps us honest: whether we succeed or fail, the predictions will be out there for all to see.
Observers can offer constructive feedback.
It gives us the opportunity post our own observations.
It offers the general public a window into the intersection of social data and political research.

So knowing this, I peeked at the putative winners for some races I’m following, starting with the competitive bout between Democratic incumbent Lois Capps and challenger Abel Maldonado in California’s 24th, Pacific Standard’s home turf. As of last night, VWYT gives it to Maldonado, with 53 percent of the vote.

Then there’s Berman-Sherman, the heated race between two Democratic incumbents battling for a single seat in LA? Ah nothing, not even Brad Sherman’s name, since the algorithm only codes for races with a Republican-Democratic contest.

OK, how about Kansas’s 2nd, the district where my mom is buried? (Kansas not being Chicago or Louisiana, Mom can’t vote there.) Incumbent Republican Lynn Jenkins is shellacking Tobias Schlingensiepen with a 65 percent prediction. That’s not surprising in this very blue state, but I wonder if the challenger’s name isn’t a bit Twitter-unfriendly.

As suggested, Huberty and co. see lots of ways this experiment can go pear-shaped, many of the potential problems hinging on using 2010 results to craft important topic choices this year. The mid-terms, after all, saw a powerful effort by Tea Party partisans, which could mean the terms over-represent Republican memes and challengers’ chops. Plus, there are new issues in the mix now, such as Libya and Solyndra.

Nonetheless, it’s fun to see political science in action and even better that for once it’s not about those two guys running for president.

Michael Todd
Most of Michael Todd's career has been spent in newspaper journalism, ranging from papers in the Marshall Islands to tiny California farming communities. Before joining the publishing arm of the Miller-McCune Center, he was managing editor of the national magazine Hispanic Business.

More From Michael Todd

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

September 22 • 6:00 AM

Zombies in the Quad: The Trouble With Elite Education

William Deresiewicz’s new book, Excellent Sheep, is in part, he says, a letter to his younger, more privileged self.


September 22 • 4:02 AM

You’re Going to Die! So Buy Now!

New research finds inserting reminders of our mortality into advertisements is a surprisingly effective strategy to sell products.



September 19 • 4:00 PM

In Your Own Words: What It’s Like to Get Sued Over Past Debts

Some describe their surprise when they were sued after falling behind on medical and credit card bills.



September 19 • 1:26 PM

For Charitable Products, Sex Doesn’t Sell

Sexy women may turn heads, but for pro-social and charitable products, they won’t change minds.


September 19 • 12:00 PM

Carbon Taxes Really Do Work

A new study shows that taxing carbon dioxide emissions could actually work to reduce greenhouse gases without any negative effects on employment and revenues.


September 19 • 10:00 AM

Why the Poor Remain Poor

A follow-up to “How Being Poor Makes You Poor.”


September 19 • 9:03 AM

Why Science Won’t Defeat Ebola

While science will certainly help, winning the battle against Ebola is a social challenge.


September 19 • 8:00 AM

Burrito Treason in the Lone Star State

Did Meatless Mondays bring down Texas Agriculture Commissioner Todd Staples?


September 19 • 7:31 AM

Savor Good Times, Get Through the Bad Ones—With Categories

Ticking off a category of things to do can feel like progress or a fun time coming to an end.


September 19 • 6:00 AM

The Most Untouchable Man in Sports

How the head of the governing body for the world’s most popular sport freely wields his wildly incompetent power.


September 19 • 4:00 AM

The Danger of Dining With an Overweight Companion

There’s a good chance you’ll eat more unhealthy food.



September 18 • 4:00 PM

Racial Disparity in Imprisonment Inspires White People to Be Even More Tough on Crime

White Americans are more comfortable with punitive and harsh policing and sentencing when they imagine that the people being policed and put in prison are black.



September 18 • 2:00 PM

The Wages of Millions Are Being Seized to Pay Past Debts

A new study provides the first-ever tally of how many employees lose up to a quarter of their paychecks over debts like unpaid credit card or medical bills and student loans.


September 18 • 12:00 PM

When Counterfeit and Contaminated Drugs Are Deadly

The cost and the crackdown, worldwide.


September 18 • 10:00 AM

How Do You Make a Living, Molly Crabapple?

Noah Davis talks to Molly Crapabble about Michelangelo, the Medicis, and the tension between making art and making money.


September 18 • 9:00 AM

Um, Why Are These Professors Creeping on My Facebook Page?

The ethics of student-teacher “intimacy”—on campus and on social media.


September 18 • 8:00 AM

Welcome to the Economy Economy

With the recent introduction of Apple Pay, the Silicon Valley giant is promising to remake how we interact with money. Could iCoin be next?



September 18 • 6:09 AM

How to Build a Better Election

Elimination-style voting is harder to fiddle with than majority rule.


September 18 • 6:00 AM

Homeless on Purpose

The latest entry in a series of interviews about subculture in America.


September 18 • 4:00 AM

Why Original Artworks Move Us More Than Reproductions

Researchers present evidence that hand-created artworks convey an almost magical sense of the artist’s essence.


Follow us


For Charitable Products, Sex Doesn’t Sell

Sexy women may turn heads, but for pro-social and charitable products, they won't change minds.

Carbon Taxes Really Do Work

A new study shows that taxing carbon dioxide emissions could actually work to reduce greenhouse gases without any negative effects on employment and revenues.

Savor Good Times, Get Through the Bad Ones—With Categories

Ticking off a category of things to do can feel like progress or a fun time coming to an end.

How to Build a Better Election

Elimination-style voting is harder to fiddle with than majority rule.

Do Conspiracy Theorists Feed on Unsuspecting Internet Trolls?

Not literally, but debunkers and satirists do fuel conspiracy theorists' appetites.

The Big One

One in three tourists to Jamaica reports getting harassed; half of them are hassled to buy drugs. September/October 2014 new-big-one-4

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.