Menus Subscribe Search

Follow us


The World Wide Web

secret-service

(Photo: David Stuart Productions/Shutterstock)

Can We Really Detect Sarcasm With a Machine?

• June 17, 2014 • 10:00 AM

(Photo: David Stuart Productions/Shutterstock)

What was once the domain of literary critics has now become the world of the Secret Service.

Eyes rolled a couple of weeks ago when the Secret Service posted a work order for new social media analytics software. Not because the law enforcement agency tasked with protecting the president and other federal officials lacks a social media presence, but because among the 22 functionality requirements for the software was the “ability to detect sarcasm and false positives.”

Edwin M. Donovan, deputy assistant director of the Secret Service’s Office of Government and Public Affairs, emphasizes that sarcasm detection is only one of many requirements. “It’s so you don’t have to sift through thousands of tweets,” he says. “We want to streamline, to automate the social media monitoring process. Not just for sarcasm.”

“Right now, if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.”

Donovan says that assessing large volumes of messages has been a problem for the Secret Service, which in the past has borrowed the analytics tools of other agencies, like FEMA. “It’s an issue that’s come up before, and we don’t want to waste time sifting through messages.”

“Right now,” Donovan says, “if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.” The agency wants to be able to monitor its own presence on social media, but also track trending topics and influential users.

“Remember the purple tunnel of doom?” Donovan asks, referring to the debacle during the 2009 presidential inauguration when thousands of attendances with purple tickets were caught in the Third Street Tunnel, unable to cross 395 to attend the event. “We weren’t monitoring Twitter that day, so we didn’t know. Since then, we’ve entered social media.”

The Secret Service (@SecretService) joined Twitter in February of 2010. The account has posted only 572 tweets, but is followed by 110,000 users. And it’s not the only governmental agency interested in assessing the tone of other users. Last summer, the BBC reported that the French company Spotter provided such an analytics tool to the British Home Office, the European Union Commission, and Dubai Courts.

For around $1,675 per month, Spotter provides software that “uses a combination of linguistics, semantics and heuristics to create algorithms that generate reports about online reputation.” Determining whether users are genuinely complaining or sincerely threatening a government agency is difficult, but Spotter claims an 80 percent accuracy rate and offers the service in 29 languages.

But how does software accomplish a task that is difficult even for sentient beings? Even the most casual of correspondents has had a text message or email misunderstood by its recipient. Without the aid of eye rolls or shoulder shrugs, textual sarcasm can be difficult to detect. The science of sincerity, or sarcasm if you’re a glass-empty type, is critical for law enforcement agencies, but also profitable for corporations seeking to improve their customer service. Understanding whether a reviewer sarcastically praised your product or honestly questioned your services is critical.

So what used to be the domain of literary critics has become a novel problem for software designers. A 2010 paper by three researchers at Hebrew University presented a sarcasm algorithm with a 77 percent precision rate. Developed through an analysis of 66,000 Amazon product reviews, the algorithm was able to detect the situational irony of titles (“[I] Love the Cover” for a book review and “Where Am I?” for a GPS device) and sarcastic patterns in speech and punctuation (sentence length, multiple exclamation or question marks in sentences, and the number of words with all capital letters among others).

But Amazon reviews have many words. In a 2011 paper, three researchers at Rutgers’ School of Communication & Information were much less successful at identifying sarcasm on Twitter, which is limited to 140-character tweets. Tweet length as well as the lack of context made assessing tone very difficult, for machines but also humans. Even with the assistance of Twitter tics like emoticons and hashtags, the human judges and the machines demonstrated only 70 percent accuracy when distinguishing sarcastic tweets from positives or negative tweets.

But that was 2011, and with companies like Spotter already reporting 80 percent accuracy, the Secret Service will likely find more bids than it needs. Donovan expressed some sarcasm of his own when I asked about how they will measure the actual accuracy of whichever software they purchase. “That’s a great question,” Donovan says, “for the companies that make these claims.”

Casey N. Cep
Casey N. Cep is a writer from the Eastern Shore of Maryland. She has written for the New Republic, the New York Times, the New Yorker, and the Paris Review. Follow her on Twitter @cncep.

More From Casey N. Cep

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

November 26 • 12:00 PM

How Do You Make a Living, Taxidermist?

Taxidermist Katie Innamorato talks to Noah Davis about learning her craft, seeing it become trendy, and the going-rate for a “Moss Fox.”


November 26 • 10:28 AM

Attitudes About Race Affect Actions, Even When They Don’t

Tiny effects of attitudes on individuals’ actions pile up quickly.


November 26 • 10:13 AM

Honeybees Touring America


November 26 • 10:00 AM

Understanding Money

In How to Speak Money, John Lanchester explains how the monied people talk about their mountains of cash.


November 26 • 8:00 AM

The Exponential Benefits of Eating Less

Eating less food—whole food and junk food, meat and plants, organic and conventional, GMO and non-GMO—would do a lot more than just better our personal health.


November 26 • 6:00 AM

The Incorruptible Bodies of Saints

Their figures were helped along by embalming, but, somehow, everyone forgot that part.


November 26 • 4:00 AM

The Geography of Real Estate Markets Is Shifting Under Our Feet

Policies aimed at unleashing supply in order to make housing more affordable are relying on outdated models.



November 25 • 4:00 PM

Is the Federal Reserve Bank of New York Doing Enough to Monitor Wall Street?

Bank President William Dudley says supervision is stronger than ever, but Democratic senators are unconvinced: “You need to fix it, Mr. Dudley, or we need to get someone who will.”


November 25 • 3:30 PM

Cultural Activities Help Seniors Retain Health Literacy

New research finds a link between the ability to process health-related information and regular attendance at movies, plays, and concerts.


November 25 • 12:00 PM

Why Did Doctors Stop Giving Women Orgasms?

You can thank the rise of the vibrator for that, according to technology historian Rachel Maines.


November 25 • 10:08 AM

Geography, Race, and LOLs

The online lexicon spreads through racial and ethnic groups as much as it does through geography and other traditional linguistic measures.


November 25 • 10:00 AM

If It’s Yellow, Seriously, Let It Mellow

If you actually care about water and the future of the species, you’ll think twice about flushing.


November 25 • 8:00 AM

Sometimes You Should Just Say No to Surgery

The introduction of national thyroid cancer screening in South Korea led to a 15-fold increase in diagnoses and a corresponding explosion of operations—but no difference in mortality rates. This is a prime example of over-diagnosis that’s contributing to bloated health care costs.



November 25 • 6:00 AM

The Long War Between Highbrow and Lowbrow

Despise The Avengers? Loathe the snobs who despise The Avengers? You’re not the first.


November 25 • 4:00 AM

Are Women More Open to Sex Than They Admit?

New research questions the conventional wisdom that men overestimate women’s level of sexual interest in them.


November 25 • 2:00 AM

The Geography of Innovation, or, Why Almost All Japanese People Hate Root Beer

Innovation is not a product of population density, but of something else entirely.


November 24 • 4:00 PM

Federal Reserve Announces Sweeping Review of Its Big Bank Oversight

The Federal Reserve Board wants to look at whether the views of examiners are being heard by higher-ups.



November 24 • 2:00 PM

That Catcalling Video Is a Reminder of Why Research Methods Are So Important

If your methods aren’t sound then neither are your findings.


November 24 • 12:00 PM

Yes, Republicans Can Still Win the White House

If the economy in 2016 is where it was in 2012 or better, Democrats will likely retain the White House. If not, well….


November 24 • 11:36 AM

Feeling—Not Being—Wealthy Cuts Support for Economic Redistribution

A new study suggests it’s relative wealth that leads people to oppose taxing the rich and giving to the poor.


November 24 • 10:00 AM

Why Are Patients Drawn to Certain Doctors?

We look for an emotional fit between our physicians and ourselves—and right now, that’s the best we can do.


November 24 • 8:00 AM

Why Do We Elect Corrupt Politicians?

Voters, it seems, are willing to forgive—over and over again—dishonest yet beloved politicians if they think the job is still getting done.


Follow us


Attitudes About Race Affect Actions, Even When They Don’t

Tiny effects of attitudes on individuals' actions pile up quickly.

Geography, Race, and LOLs

The online lexicon spreads through racial and ethnic groups as much as it does through geography and other traditional linguistic measures.

Feeling—Not Being—Wealthy Cuts Support for Economic Redistribution

A new study suggests it's relative wealth that leads people to oppose taxing the rich and giving to the poor.

Sufferers of Social Anxiety Disorder, Your Friends Like You

The first study of friends' perceptions suggest they know something's off with their pals but like them just the same.

Standing Up for My Group by Kicking Yours

Members of a minority ethnic group are less likely to express support for gay equality if they believe their own group suffers from discrimination.

The Big One

One in two United States senators and two in five House members who left office between 1998 and 2004 became lobbyists. November/December 2014

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.