Menus Subscribe Search

Follow us


The World Wide Web

secret-service

(Photo: David Stuart Productions/Shutterstock)

Can We Really Detect Sarcasm With a Machine?

• June 17, 2014 • 10:00 AM

(Photo: David Stuart Productions/Shutterstock)

What was once the domain of literary critics has now become the world of the Secret Service.

Eyes rolled a couple of weeks ago when the Secret Service posted a work order for new social media analytics software. Not because the law enforcement agency tasked with protecting the president and other federal officials lacks a social media presence, but because among the 22 functionality requirements for the software was the “ability to detect sarcasm and false positives.”

Edwin M. Donovan, deputy assistant director of the Secret Service’s Office of Government and Public Affairs, emphasizes that sarcasm detection is only one of many requirements. “It’s so you don’t have to sift through thousands of tweets,” he says. “We want to streamline, to automate the social media monitoring process. Not just for sarcasm.”

“Right now, if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.”

Donovan says that assessing large volumes of messages has been a problem for the Secret Service, which in the past has borrowed the analytics tools of other agencies, like FEMA. “It’s an issue that’s come up before, and we don’t want to waste time sifting through messages.”

“Right now,” Donovan says, “if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.” The agency wants to be able to monitor its own presence on social media, but also track trending topics and influential users.

“Remember the purple tunnel of doom?” Donovan asks, referring to the debacle during the 2009 presidential inauguration when thousands of attendances with purple tickets were caught in the Third Street Tunnel, unable to cross 395 to attend the event. “We weren’t monitoring Twitter that day, so we didn’t know. Since then, we’ve entered social media.”

The Secret Service (@SecretService) joined Twitter in February of 2010. The account has posted only 572 tweets, but is followed by 110,000 users. And it’s not the only governmental agency interested in assessing the tone of other users. Last summer, the BBC reported that the French company Spotter provided such an analytics tool to the British Home Office, the European Union Commission, and Dubai Courts.

For around $1,675 per month, Spotter provides software that “uses a combination of linguistics, semantics and heuristics to create algorithms that generate reports about online reputation.” Determining whether users are genuinely complaining or sincerely threatening a government agency is difficult, but Spotter claims an 80 percent accuracy rate and offers the service in 29 languages.

But how does software accomplish a task that is difficult even for sentient beings? Even the most casual of correspondents has had a text message or email misunderstood by its recipient. Without the aid of eye rolls or shoulder shrugs, textual sarcasm can be difficult to detect. The science of sincerity, or sarcasm if you’re a glass-empty type, is critical for law enforcement agencies, but also profitable for corporations seeking to improve their customer service. Understanding whether a reviewer sarcastically praised your product or honestly questioned your services is critical.

So what used to be the domain of literary critics has become a novel problem for software designers. A 2010 paper by three researchers at Hebrew University presented a sarcasm algorithm with a 77 percent precision rate. Developed through an analysis of 66,000 Amazon product reviews, the algorithm was able to detect the situational irony of titles (“[I] Love the Cover” for a book review and “Where Am I?” for a GPS device) and sarcastic patterns in speech and punctuation (sentence length, multiple exclamation or question marks in sentences, and the number of words with all capital letters among others).

But Amazon reviews have many words. In a 2011 paper, three researchers at Rutgers’ School of Communication & Information were much less successful at identifying sarcasm on Twitter, which is limited to 140-character tweets. Tweet length as well as the lack of context made assessing tone very difficult, for machines but also humans. Even with the assistance of Twitter tics like emoticons and hashtags, the human judges and the machines demonstrated only 70 percent accuracy when distinguishing sarcastic tweets from positives or negative tweets.

But that was 2011, and with companies like Spotter already reporting 80 percent accuracy, the Secret Service will likely find more bids than it needs. Donovan expressed some sarcasm of his own when I asked about how they will measure the actual accuracy of whichever software they purchase. “That’s a great question,” Donovan says, “for the companies that make these claims.”

Casey N. Cep
Casey N. Cep is a writer from the Eastern Shore of Maryland. She has written for the New Republic, the New York Times, the New Yorker, and the Paris Review. Follow her on Twitter @cncep.

More From Casey N. Cep

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

October 24 • 10:00 AM

Why DNA Is One of Humanity’s Greatest Inventions

How we’ve co-opted our genetic material to change our world.


October 24 • 8:00 AM

What Do Clowns Think of Clowns?

Three major players weigh in on the current state of the clown.


October 24 • 7:13 AM

There Is No Surge in Illegal Immigration

The overall rate of illegal immigration has actually decreased significantly in the last 10 years. The time is ripe for immigration reform.


October 24 • 6:15 AM

Politicians Really Aren’t Better Decision Makers

Politicians took part in a classic choice experiment but failed to do better than the rest of us.


October 24 • 5:00 AM

Why We Gossip: It’s Really All About Ourselves

New research from the Netherlands finds stories we hear about others help us determine how we’re doing.


October 24 • 2:00 AM

Congratulations, Your City Is Dying!

Don’t take population numbers at face value.


October 23 • 4:00 PM

Of Course Marijuana Addiction Exists

The polarized legalization debate leads to exaggerated claims and denials about pot’s potential harms. The truth lies somewhere in between.


October 23 • 2:00 PM

American Companies Are Getting Way Too Cozy With the National Security Agency

Newly released documents describe “contractual relationships” between the NSA and U.S. companies, as well as undercover operatives.


October 23 • 12:00 PM

The Man Who’s Quantifying New York City

Noah Davis talks to the proprietor of I Quant NY. His methodology: a little something called “addition.”


October 23 • 11:02 AM

Earliest High-Altitude Settlements Found in Peru

Discovery suggests humans adapted to high altitude faster than previously thought.


October 23 • 10:00 AM

The Psychology of Bribery and Corruption

An FBI agent offered up confidential information about a political operative’s enemy in exchange for cash—and they both got caught. What were they thinking?


October 23 • 8:00 AM

Ebola News Gives Me a Guilty Thrill. Am I Crazy?

What it means to feel a little excited about the prospect of a horrific event.


October 23 • 7:04 AM

Why Don’t Men Read Romance Novels?

A lot of men just don’t read fiction, and if they do, structural misogyny drives them away from the genre.


October 23 • 6:00 AM

Why Do Americans Pray?

It depends on how you ask.


October 23 • 4:00 AM

Musicians Are Better Multitaskers

New research from Canada finds trained musicians more efficiently switch from one mental task to another.


October 22 • 4:00 PM

The Last Thing the Women’s Movement Needs Is a Heroic Male Takeover

Is the United Nations’ #HeForShe campaign helping feminism?


October 22 • 2:00 PM

Turning Public Education Into Private Profits

Baker Mitchell is a politically connected North Carolina businessman who celebrates the power of the free market. Every year, millions of public education dollars flow through Mitchell’s chain of four non-profit charter schools to for-profit companies he controls.


October 22 • 12:00 PM

Will the End of a Tax Loophole Kill Off Irish Business and Force Google and Apple to Pay Up?

U.S. technology giants have constructed international offices in Dublin in order to take advantage of favorable tax policies that are now changing. But Ireland might have enough other draws to keep them there even when costs climb.


October 22 • 10:00 AM

Veterans in the Ivory Tower

Why there aren’t enough veterans at America’s top schools—and what some people are trying to do to change that.


October 22 • 8:00 AM

Our Language Prejudices Don’t Make No Sense

We should embrace the fact that there’s no single recipe for English. Making fun of people for replacing “ask” with “aks,” or for frequently using double negatives just makes you look like the unsophisticated one.


October 22 • 7:04 AM

My Politicians Are Better Looking Than Yours

A new study finds we judge the cover by the book—or at least the party.


October 22 • 6:00 AM

How We Form Our Routines

Whether it’s a morning cup of coffee or a glass of warm milk before bed, we all have our habitual processions. The way they become engrained, though, varies from person to person.


October 22 • 4:00 AM

For Preschoolers, Spite and Smarts Go Together

New research from Germany finds greater cognitive skills are associated with more spiteful behavior in children.


October 21 • 4:00 PM

Why the Number of Reported Sexual Offenses Is Skyrocketing at Occidental College

When you make it easier to report assault, people will come forward.


October 21 • 2:00 PM

Private Donors Are Supplying Spy Gear to Cops Across the Country Without Any Oversight

There’s little public scrutiny when private donors pay to give police controversial technology and weapons. Sometimes, companies are donors to the same foundations that purchase their products for police.


Follow us


Politicians Really Aren’t Better Decision Makers

Politicians took part in a classic choice experiment but failed to do better than the rest of us.

Earliest High-Altitude Settlements Found in Peru

Discovery suggests humans adapted to high altitude faster than previously thought.

My Politicians Are Better Looking Than Yours

A new study finds we judge the cover by the book—or at least the party.

That Cigarette Would Make a Great Water Filter

Clean out the ashtray, add some aluminum oxide, and you've (almost) got yourself a low-cost way to remove arsenic from drinking water.

Love and Hate in Israel and Palestine

Psychologists find that parties to a conflict think they're motivated by love while their enemies are motivated by hate.

The Big One

One company, Amazon, controls 67 percent of the e-book market in the United States—down from 90 percent five years ago. September/October 2014 new-big-one-5

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.