Menus Subscribe Search

Predicting House Races by Weight of Tweet

• October 31, 2012 • 4:00 AM

With the presidential prediction game dreadfully same-y, surely there’s another constantly changing fix for political junkies. How about forecasting all 435 U.S. congressional races every day, based on brand new data every day?

That’s what you get at “Voting With Your Tweet,” an experiment that mines mentions of congressional candidates in the Twitter-sphere to predict who will win each race and what the actual vote share will be. Unlike past efforts at using social media to predict political contests, which made their “predictions” after the voters had settled the matter, this research is happening now and you can check in at the California News Service site to see what’s up today, yesterday, or tomorrow.

Using social media as a forecasting tool is a hot area right now, and mixing Twitter with punditry is not brand new. Carnegie Mellon’s Brendan O’Connor compared tweeting to polling results in 2010, and not surprisingly found some correlation. Andranik Tumasjan and his colleagues at the Munich Technical University found that “the mere number of messages mentioning a party reflects the election result” in a German federal election. The “media utility” Tweetminster, meanwhile, says it accurately predicted the outcome of the last British Parliamentary elections using Twitter (an experiment they note is based on an earlier Japanese study looking at “online buzz and election results”).

Voting With Your Tweet is a kind of combination, using past results to predict future returns (but with mountains of caveats).

Analyzing about a quarter million tweets from 356 races in the 2010 mid-terms, UC Berkeley doctoral candidate Mark Huberty correctly “predicted” (after the fact) the winners 92 percent of the time, a much better showing than the six professional human pundits used for comparison. (The technical report on 2010 is available here, but be prepared to see that the 92 percent figure is one of a number of outcomes based on different statistical methods and statistical approaches. Huberty goes with “better than 85 percent accuracy” in the paper’s introduction.)

Taking that information, Huberty “trained two machine learning algorithms to determine what word features of those tweets best predicted whether the Democratic or Republican party candidate won each race.” So in its current incarnation …

We think the finished algorithm works like this:

First, it identifies from the language in a candidate’s tweets whether they are the incumbent or challenger. Since incumbents win about 85% of the time, this provides a good baseline.

It then adjusts the baseline prediction based on sentiment and action-related phrases. For instance, “voted hcr” (indicating that the incumbent voted for health care reform) was one of the most influential predictors alongside incumbency-related phrases. The algorithm weights those phrases positively or negatively, depending on how predictive they were of a candidate winning or losing.

And you can take this forecast to the bank? Only if you’re Lehman Brothers. “Voting with your Tweet is an experiment and should be treated as such,” Huberty and “data guru” Len DeGroot from the Knight Digital Media Center at Berkeley’s Graduate School of Journalism write in an honest and detailed set of FAQs. “We think this might work and think we might know why. But we could fail spectacularly.”

So why do it at all, or at least so publicly? The answers are refreshing:

Anyone who is interested can observe the experiment.
It keeps us honest: whether we succeed or fail, the predictions will be out there for all to see.
Observers can offer constructive feedback.
It gives us the opportunity post our own observations.
It offers the general public a window into the intersection of social data and political research.

So knowing this, I peeked at the putative winners for some races I’m following, starting with the competitive bout between Democratic incumbent Lois Capps and challenger Abel Maldonado in California’s 24th, Pacific Standard’s home turf. As of last night, VWYT gives it to Maldonado, with 53 percent of the vote.

Then there’s Berman-Sherman, the heated race between two Democratic incumbents battling for a single seat in LA? Ah nothing, not even Brad Sherman’s name, since the algorithm only codes for races with a Republican-Democratic contest.

OK, how about Kansas’s 2nd, the district where my mom is buried? (Kansas not being Chicago or Louisiana, Mom can’t vote there.) Incumbent Republican Lynn Jenkins is shellacking Tobias Schlingensiepen with a 65 percent prediction. That’s not surprising in this very blue state, but I wonder if the challenger’s name isn’t a bit Twitter-unfriendly.

As suggested, Huberty and co. see lots of ways this experiment can go pear-shaped, many of the potential problems hinging on using 2010 results to craft important topic choices this year. The mid-terms, after all, saw a powerful effort by Tea Party partisans, which could mean the terms over-represent Republican memes and challengers’ chops. Plus, there are new issues in the mix now, such as Libya and Solyndra.

Nonetheless, it’s fun to see political science in action and even better that for once it’s not about those two guys running for president.

Michael Todd
Most of Michael Todd's career has been spent in newspaper journalism, ranging from papers in the Marshall Islands to tiny California farming communities. Before joining the publishing arm of the Miller-McCune Center, he was managing editor of the national magazine Hispanic Business.

More From Michael Todd

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

August 20 • 4:00 PM

Why Can’t Conservatives See the Benefits of Affordable Child Care?

Private programs might do a better job of watching our kids than state-run programs, but they’re not accessible to everyone.


August 20 • 2:00 PM

Oil and Gas Companies Are Illegally Using Diesel Fuel in Hundreds of Fracking Operations

An analysis by an environmental group finds hundreds of cases in which drillers used diesel fuel without obtaining permits and sometimes altered records disclosing they had done so.


August 20 • 12:00 PM

The Mystery of Britain’s Alien Big Cats

In a nation where the biggest carnivorous predator is a badger, why are there so many reported sightings of large cats?


August 20 • 10:00 AM

Death Row in Arizona: Where Human Experimentation Is the Rule, Not the Exception

Recent reports show that chemical roulette is the state’s M.O.


August 20 • 9:51 AM

Diversity Is in the Eye of the Beholder

Perception of group diversity depends on the race of the observer and the extent to which they worry about discrimination.


August 20 • 8:40 AM

Psychopathic or Just Antisocial? A Key Brain Difference Tells the Tale

Though psychopaths and antisocial people may seem similar, what occurs in their brains isn’t.


August 20 • 8:00 AM

What the Cost of Raising a Child in America Tells Us About Income Inequality

You’ll spend nearly a quarter of a million dollars to raise a kid in the United States, or about five times the annual median income.


August 20 • 6:00 AM

In Praise of ‘American Greed’

While it remains semi-hidden on CNBC and can’t claim the car chases of Cops, American Greed—now with eight seasons in the books—has proven itself a worthy endeavor.


August 20 • 4:00 AM

Of Course I Behaved Like a Jerk, I Was Just Watching ‘Jersey Shore’

Researchers find watching certain types of reality TV can make viewers more aggressive.


August 20 • 2:00 AM

Concluding Remarks About Housing Affordability and Supply Restricitions

Demand, not supply, plays the dominant role in explaining the housing affordability crisis. The wages are just too damn low.


August 19 • 4:00 PM

Can Lawmakers Only Make Laws That Corporations Allow?

There’s a telling detail in a recent story about efforts to close loopholes in corporate tax laws.




August 19 • 12:00 PM

How ‘Contagion’ Became Contagious

Do ideas and emotions really spread like a virus?


August 19 • 10:00 AM

Child Refugees: The New Barbarians

The disturbing rhetoric around the recent rise in child refugees into the United States from Central America may be shaping popular opinion on upcoming immigration reform.


August 19 • 8:00 AM

Making Police Departments More Diverse Isn’t Enough

Local police departments should reflect the communities they serve, but fixing that alone won’t curb unnecessary violence.


August 19 • 7:15 AM

Common Knowledge Makes Us More Cooperative

People are more inclined to take mutually beneficial risks if they know what others know.


August 19 • 6:00 AM

Seeking a Healthy Public School Lunch? Good Luck

Mystery meat will always win.


August 19 • 4:00 AM

The Positive Effects of Sports-Themed Video Games

New research finds sports-themed video games actually encourage some kids to get onto the field.


August 19 • 1:00 AM

DIY Diagnosis: How an Extreme Athlete Uncovered Her Genetic Flaw

When Kim Goodsell discovered that she had two extremely rare genetic diseases, she taught herself genetics to help find out why.



August 18 • 3:30 PM

Mister Rogers’ Heart-Healthy Neighborhood

Researchers find living in a friendly, cohesive neighborhood lowers seniors’ chances of having a heart attack.


August 18 • 2:00 PM

Wealth or Good Parenting?

Framing the privileges of the rich.


August 18 • 12:00 PM

How Much Did the Stigma of Mental Illness Harm Robin Williams?

Addiction treatment routinely fails people with mental illnesses, while mental health care often ignores addiction. And everywhere, stigma is rife. Can a tragic death prompt a more intelligent approach?


August 18 • 10:00 AM

Punished for Being Poor: The Problem With Using Big Data in the Justice System

Correctional departments use data-driven analyses because they’re easier and cheaper than individual assessments. But at what cost?


Follow us


Diversity Is in the Eye of the Beholder

Perception of group diversity depends on the race of the observer and the extent to which they worry about discrimination.

Psychopathic or Just Antisocial? A Key Brain Difference Tells the Tale

Though psychopaths and antisocial people may seem similar, what occurs in their brains isn’t.

Common Knowledge Makes Us More Cooperative

People are more inclined to take mutually beneficial risks if they know what others know.

How a Shift in Human Head Shape Changed Everything

When did homo sapiens become a more sophisticated species? Not until our skulls underwent "feminization."

Journalists Can Get PTSD Without Leaving Their Desks

Dealing with violent content takes a heavy toll on some reporters.

The Big One

One in two full-time American fast-food workers' families are enrolled in public assistance programs, at a cost of $7 billion per year. July/August 2014 fast-food-big-one
Subscribe Now

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.