Menus Subscribe Search

Follow us


Predicting House Races by Weight of Tweet

• October 31, 2012 • 4:00 AM

With the presidential prediction game dreadfully same-y, surely there’s another constantly changing fix for political junkies. How about forecasting all 435 U.S. congressional races every day, based on brand new data every day?

That’s what you get at “Voting With Your Tweet,” an experiment that mines mentions of congressional candidates in the Twitter-sphere to predict who will win each race and what the actual vote share will be. Unlike past efforts at using social media to predict political contests, which made their “predictions” after the voters had settled the matter, this research is happening now and you can check in at the California News Service site to see what’s up today, yesterday, or tomorrow.

Using social media as a forecasting tool is a hot area right now, and mixing Twitter with punditry is not brand new. Carnegie Mellon’s Brendan O’Connor compared tweeting to polling results in 2010, and not surprisingly found some correlation. Andranik Tumasjan and his colleagues at the Munich Technical University found that “the mere number of messages mentioning a party reflects the election result” in a German federal election. The “media utility” Tweetminster, meanwhile, says it accurately predicted the outcome of the last British Parliamentary elections using Twitter (an experiment they note is based on an earlier Japanese study looking at “online buzz and election results”).

Voting With Your Tweet is a kind of combination, using past results to predict future returns (but with mountains of caveats).

Analyzing about a quarter million tweets from 356 races in the 2010 mid-terms, UC Berkeley doctoral candidate Mark Huberty correctly “predicted” (after the fact) the winners 92 percent of the time, a much better showing than the six professional human pundits used for comparison. (The technical report on 2010 is available here, but be prepared to see that the 92 percent figure is one of a number of outcomes based on different statistical methods and statistical approaches. Huberty goes with “better than 85 percent accuracy” in the paper’s introduction.)

Taking that information, Huberty “trained two machine learning algorithms to determine what word features of those tweets best predicted whether the Democratic or Republican party candidate won each race.” So in its current incarnation …

We think the finished algorithm works like this:

First, it identifies from the language in a candidate’s tweets whether they are the incumbent or challenger. Since incumbents win about 85% of the time, this provides a good baseline.

It then adjusts the baseline prediction based on sentiment and action-related phrases. For instance, “voted hcr” (indicating that the incumbent voted for health care reform) was one of the most influential predictors alongside incumbency-related phrases. The algorithm weights those phrases positively or negatively, depending on how predictive they were of a candidate winning or losing.

And you can take this forecast to the bank? Only if you’re Lehman Brothers. “Voting with your Tweet is an experiment and should be treated as such,” Huberty and “data guru” Len DeGroot from the Knight Digital Media Center at Berkeley’s Graduate School of Journalism write in an honest and detailed set of FAQs. “We think this might work and think we might know why. But we could fail spectacularly.”

So why do it at all, or at least so publicly? The answers are refreshing:

Anyone who is interested can observe the experiment.
It keeps us honest: whether we succeed or fail, the predictions will be out there for all to see.
Observers can offer constructive feedback.
It gives us the opportunity post our own observations.
It offers the general public a window into the intersection of social data and political research.

So knowing this, I peeked at the putative winners for some races I’m following, starting with the competitive bout between Democratic incumbent Lois Capps and challenger Abel Maldonado in California’s 24th, Pacific Standard’s home turf. As of last night, VWYT gives it to Maldonado, with 53 percent of the vote.

Then there’s Berman-Sherman, the heated race between two Democratic incumbents battling for a single seat in LA? Ah nothing, not even Brad Sherman’s name, since the algorithm only codes for races with a Republican-Democratic contest.

OK, how about Kansas’s 2nd, the district where my mom is buried? (Kansas not being Chicago or Louisiana, Mom can’t vote there.) Incumbent Republican Lynn Jenkins is shellacking Tobias Schlingensiepen with a 65 percent prediction. That’s not surprising in this very blue state, but I wonder if the challenger’s name isn’t a bit Twitter-unfriendly.

As suggested, Huberty and co. see lots of ways this experiment can go pear-shaped, many of the potential problems hinging on using 2010 results to craft important topic choices this year. The mid-terms, after all, saw a powerful effort by Tea Party partisans, which could mean the terms over-represent Republican memes and challengers’ chops. Plus, there are new issues in the mix now, such as Libya and Solyndra.

Nonetheless, it’s fun to see political science in action and even better that for once it’s not about those two guys running for president.

Michael Todd
Most of Michael Todd's career has been spent in newspaper journalism, ranging from papers in the Marshall Islands to tiny California farming communities. Before joining the publishing arm of the Miller-McCune Center, he was managing editor of the national magazine Hispanic Business.

More From Michael Todd

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

December 18 • 4:00 PM

How I Navigated Life as a Newly Sober Mom

Saying “no” to my kids was harder than saying “no” to alcohol. But for their sake and mine, I had to learn to put myself first sometimes.


December 18 • 2:00 PM

Women in Apocalyptic Fiction Shaving Their Armpits

Because our interest in realism apparently only goes so far.


December 18 • 12:00 PM

The Paradox of Choice, 10 Years Later

Paul Hiebert talks to psychologist Barry Schwartz about how modern trends—social media, FOMO, customer review sites—fit in with arguments he made a decade ago in his highly influential book, The Paradox of Choice: Why More Is Less.


December 18 • 10:00 AM

What It’s Like to Spend a Few Hours in the Church of Scientology

Wrestling with thetans, attempting to unlock a memory bank, and a personality test seemingly aimed at people with depression. This is Scientology’s “dissemination drill” for potential new members.


December 18 • 8:00 AM

Gendering #BlackLivesMatter: A Feminist Perspective

Black men are stereotyped as violent, while black women are rendered invisible. Here’s why the gendering of black lives matters.


December 18 • 7:06 AM

Apparently You Can Bring Your Religion to Work

New research says offices that encourage talk of religion actually make for happier workplaces.


December 18 • 6:00 AM

The Very Weak and Complicated Links Between Mental Illness and Gun Violence

Vanderbilt University’s Jonathan Metzl and Kenneth MacLeish address our anxieties and correct our assumptions.


December 18 • 4:00 AM

Should Movies Be Rated RD for Reckless Driving?

A new study finds a link between watching films featuring reckless driving and engaging in similar behavior years later.


December 17 • 4:00 PM

How to Run a Drug Dealing Network in Prison

People tend not to hear about the prison drug dealing operations that succeed. Substance.com asks a veteran of the game to explain his system.


December 17 • 2:00 PM

Gender Segregation of Toys Is on the Rise

Charting the use of “toys for boys” and “toys for girls” in American English.


December 17 • 12:41 PM

Why the College Football Playoff Is Terrible But Better Than Before

The sample size is still embarrassingly small, but at least there’s less room for the availability cascade.


December 17 • 11:06 AM

Canadian Kids Have a Serious Smoking Problem

Bootleg cigarette sales could be leading Canadian teens to more serious drugs, a recent study finds.


December 17 • 10:37 AM

A Public Lynching in Sproul Plaza

When photographs of lynching victims showed up on a hallowed site of democracy in action, a provocation was issued—but to whom, by whom, and why?


December 17 • 8:00 AM

What Was the Job?

This was the year the job broke, the year we accepted a re-interpretation of its fundamental bargain and bought in to the push to get us to all work for ourselves rather than each other.


December 17 • 6:00 AM

White Kids Will Be Kids

Even the “good” kids—bound for college, upwardly mobile—sometimes break the law. The difference? They don’t have much to fear. A professor of race and social movements reflects on her teenage years and faces some uncomfortable realities.



December 16 • 4:00 PM

How Fear of Occupy Wall Street Undermined the Red Cross’ Sandy Relief Effort

Red Cross responders say there was a ban on working with the widely praised Occupy Sandy relief group because it was seen as politically unpalatable.


December 16 • 3:30 PM

Murder! Mayhem! And That’s Just the Cartoons!

New research suggests deaths are common features of animated features aimed at children.


December 16 • 1:43 PM

In Tragedy, Empathy Still Dependent on Proximity

In spite of an increasingly connected world, in the face of adversity, a personal touch is most effective.


December 16 • 12:00 PM

The ‘New York Times’ Is Hooked on Drug du Jour Journalism

For the paper of record, addiction is always about this drug or that drug rather than the real causes.


December 16 • 10:00 AM

What Is the Point of Academic Books?

Ultimately, they’re meant to disseminate knowledge. But their narrow appeal makes them expensive to produce and harder to sell.


December 16 • 8:00 AM

Unjust and Unwell: The Racial Issues That Could Be Affecting Your Health Care

Physicians and medical students have the same problems with implicit bias as the rest of us.


December 16 • 6:00 AM

If You Get Confused Just Listen to the Music Play

Healing the brain with the Grateful Dead.


December 16 • 4:00 AM

Another Casualty of the Great Recession: Trust

Research from Britain finds people who were laid off from their jobs expressed lower levels of generalized trust.


December 15 • 4:00 PM

When Charter Schools Are Non-Profit in Name Only

Some charters pass along nearly all their money to for-profit companies hired to manage the schools. It’s an arrangement that’s raising eyebrows.


Follow us


Apparently You Can Bring Your Religion to Work

New research says offices that encourage talk of religion actually make for happier workplaces.

Canadian Kids Have a Serious Smoking Problem

Bootleg cigarette sales could be leading Canadian teens to more serious drugs, a recent study finds.

The Hidden Psychology of the Home Ref

That old myth of home field bias isn’t a myth at all; it’s a statistical fact.

A Word of Caution to the Holiday Deal-Makers

Repeat customers—with higher return rates and real bargain-hunting prowess—can have negative effects on a company’s net earnings.

Crowdfunding Works for Science

Scientists just need to put forth some effort.

The Big One

One in two United States senators and two in five House members who left office between 1998 and 2004 became lobbyists. November/December 2014

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.