Menus Subscribe Search

Follow us


The World Wide Web

secret-service

(Photo: David Stuart Productions/Shutterstock)

Can We Really Detect Sarcasm With a Machine?

• June 17, 2014 • 10:00 AM

(Photo: David Stuart Productions/Shutterstock)

What was once the domain of literary critics has now become the world of the Secret Service.

Eyes rolled a couple of weeks ago when the Secret Service posted a work order for new social media analytics software. Not because the law enforcement agency tasked with protecting the president and other federal officials lacks a social media presence, but because among the 22 functionality requirements for the software was the “ability to detect sarcasm and false positives.”

Edwin M. Donovan, deputy assistant director of the Secret Service’s Office of Government and Public Affairs, emphasizes that sarcasm detection is only one of many requirements. “It’s so you don’t have to sift through thousands of tweets,” he says. “We want to streamline, to automate the social media monitoring process. Not just for sarcasm.”

“Right now, if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.”

Donovan says that assessing large volumes of messages has been a problem for the Secret Service, which in the past has borrowed the analytics tools of other agencies, like FEMA. “It’s an issue that’s come up before, and we don’t want to waste time sifting through messages.”

“Right now,” Donovan says, “if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.” The agency wants to be able to monitor its own presence on social media, but also track trending topics and influential users.

“Remember the purple tunnel of doom?” Donovan asks, referring to the debacle during the 2009 presidential inauguration when thousands of attendances with purple tickets were caught in the Third Street Tunnel, unable to cross 395 to attend the event. “We weren’t monitoring Twitter that day, so we didn’t know. Since then, we’ve entered social media.”

The Secret Service (@SecretService) joined Twitter in February of 2010. The account has posted only 572 tweets, but is followed by 110,000 users. And it’s not the only governmental agency interested in assessing the tone of other users. Last summer, the BBC reported that the French company Spotter provided such an analytics tool to the British Home Office, the European Union Commission, and Dubai Courts.

For around $1,675 per month, Spotter provides software that “uses a combination of linguistics, semantics and heuristics to create algorithms that generate reports about online reputation.” Determining whether users are genuinely complaining or sincerely threatening a government agency is difficult, but Spotter claims an 80 percent accuracy rate and offers the service in 29 languages.

But how does software accomplish a task that is difficult even for sentient beings? Even the most casual of correspondents has had a text message or email misunderstood by its recipient. Without the aid of eye rolls or shoulder shrugs, textual sarcasm can be difficult to detect. The science of sincerity, or sarcasm if you’re a glass-empty type, is critical for law enforcement agencies, but also profitable for corporations seeking to improve their customer service. Understanding whether a reviewer sarcastically praised your product or honestly questioned your services is critical.

So what used to be the domain of literary critics has become a novel problem for software designers. A 2010 paper by three researchers at Hebrew University presented a sarcasm algorithm with a 77 percent precision rate. Developed through an analysis of 66,000 Amazon product reviews, the algorithm was able to detect the situational irony of titles (“[I] Love the Cover” for a book review and “Where Am I?” for a GPS device) and sarcastic patterns in speech and punctuation (sentence length, multiple exclamation or question marks in sentences, and the number of words with all capital letters among others).

But Amazon reviews have many words. In a 2011 paper, three researchers at Rutgers’ School of Communication & Information were much less successful at identifying sarcasm on Twitter, which is limited to 140-character tweets. Tweet length as well as the lack of context made assessing tone very difficult, for machines but also humans. Even with the assistance of Twitter tics like emoticons and hashtags, the human judges and the machines demonstrated only 70 percent accuracy when distinguishing sarcastic tweets from positives or negative tweets.

But that was 2011, and with companies like Spotter already reporting 80 percent accuracy, the Secret Service will likely find more bids than it needs. Donovan expressed some sarcasm of his own when I asked about how they will measure the actual accuracy of whichever software they purchase. “That’s a great question,” Donovan says, “for the companies that make these claims.”

Casey N. Cep
Casey N. Cep is a writer from the Eastern Shore of Maryland. She has written for the New Republic, the New York Times, the New Yorker, and the Paris Review. Follow her on Twitter @cncep.

More From Casey N. Cep

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

October 20 • 4:00 PM

The Bird Hat Craze That Sparked a Preservation Movement

How a fashion statement at the turn of the 19th century led to the creation of the first Audubon societies.


October 20 • 2:00 PM

The Risk of Getting Killed by the Police If You Are White, and If You Are Black

An analysis of killings by police shows outsize risk for young black males.


October 20 • 12:00 PM

Love and Hate in Israel and Palestine

Psychologists find that parties to a conflict think they’re motivated by love while their enemies are motivated by hate.


October 20 • 11:00 AM

My Dog Comes First: The Importance of Pets to Homeless Youth

Dogs and cats have both advantages and disadvantages for street-involved youth.


October 20 • 10:00 AM

Homophobia Is Not a Thing of the Past

Despite growing support for LGBT rights and recent decisions from the Supreme Court regarding the legality of same-sex marriage, the battle for acceptance has not yet been decided.


October 20 • 8:00 AM

Big Boobs Matter Most

Medical mnemonics are often scandalous and sexist, but they help the student to both remember important facts and cope with challenging new experiences.


October 20 • 6:00 AM

When Disease Becomes Political: The Likely Electoral Fallout From Ebola

Will voters blame President Obama—and punish Democrats in the upcoming mid-term elections—for a climate of fear?


October 20 • 4:00 AM

Coming Soon: The Anatomy of Ignorance


October 17 • 4:00 PM

What All Military Families Need to Know About High-Cost Lenders

Lessons from over a year on the beat.


October 17 • 2:00 PM

The Majority of Languages Do Not Have Gendered Pronouns

A world without “he.” Or “she.”


October 17 • 11:01 AM

How to Water a Farm in Sandy Ground

Physicists investigate how to grow food more efficiently in fine-grained soil.


October 17 • 10:00 AM

Can Science Fiction Spur Science Innovation?

Without proper funding, the answer might not even matter.


October 17 • 8:00 AM

Seattle, the Incredible Shrinking City

Seattle is leading the way in the micro-housing movement as an affordable alternative to high-cost city living.


October 17 • 6:00 AM

‘Voodoo Death’ and How the Mind Harms the Body

Can an intense belief that you’re about to die actually kill you? Researchers are learning more about “voodoo death” and how it isn’t limited to superstitious, foreign cultures.


October 17 • 4:00 AM

That Arts Degree Is Paying Off

A survey of people who have earned degrees in the arts find they are doing relatively well, although their education didn’t provide much guidance on managing a career.


October 16 • 4:00 PM

How (Some) Economists Are Like Doomsday Cult Members

Cognitive dissonance and clinging to paradigms even in the face of accumulated anomalous facts.


October 16 • 2:00 PM

The Latest—and Most Mysterious—Player in the Nasty Battle Over Net Neutrality

As the FCC considers how to regulate Internet providers, the telecom industry’s stealth campaign for hearts and minds encompasses everything from art installations to LOLcats.


October 16 • 12:00 PM

How Many Ads Is Too Many Ads?

The conundrum of online video advertising.


October 16 • 11:00 AM

Unlocking Consciousness

A study of vegetative patients closes in on the nature of consciousness.


October 16 • 10:00 AM

The False Promises of Higher Education

Danielle Henderson spent six years and $60,000 on college and beyond. The effects of that education? Not as advertised.


October 16 • 8:00 AM

Faster Justice, Closer to Home: The Power of Community Courts

Community courts across the country are fighting judicial backlog and lowering re-arrest rates.


October 16 • 6:00 AM

Killing Your Husband to Save Yourself

Without proper legal instruments, women with abusive partners are often forced to make a difficult choice: kill or be killed.


October 16 • 4:00 AM

Personality Traits Linked to Specific Diseases

New research finds neurotic people are more likely to suffer a serious health problem.


October 16 • 2:00 AM

Comparing Apples to the Big Apple: Yes, Washington, D.C., Is More Expensive Than New York City

Why shouldn’t distant locales tied to jobs in the urban core count in a housing expenditure study?


October 15 • 4:00 PM

Why Asian American Parents Are the Least Likely to Spank Their Kids

Highly educated, middle-class parents are less likely to use corporal punishment to discipline their children than less-educated, working-class, and poor parents.


Follow us


Love and Hate in Israel and Palestine

Psychologists find that parties to a conflict think they're motivated by love while their enemies are motivated by hate.

How to Water a Farm in Sandy Ground

Physicists investigate how to grow food more efficiently in fine-grained soil.

Unlocking Consciousness

A study of vegetative patients closes in on the nature of consciousness.

Advice for Emergency Alert Systems: Don’t Cry Wolf

A survey finds college students don't always take alerts seriously.

Brain’s Reward Center Does More Than Manage Rewards

Nucleus accumbens tracks many different connections in the world, a new rat study suggests.

The Big One

One company, Amazon, controls 67 percent of the e-book market in the United States—down from 90 percent five years ago. September/October 2014 new-big-one-5

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.