Menus Subscribe Search

The World Wide Web

secret-service

(Photo: David Stuart Productions/Shutterstock)

Can We Really Detect Sarcasm With a Machine?

• June 17, 2014 • 10:00 AM

(Photo: David Stuart Productions/Shutterstock)

What was once the domain of literary critics has now become the world of the Secret Service.

Eyes rolled a couple of weeks ago when the Secret Service posted a work order for new social media analytics software. Not because the law enforcement agency tasked with protecting the president and other federal officials lacks a social media presence, but because among the 22 functionality requirements for the software was the “ability to detect sarcasm and false positives.”

Edwin M. Donovan, deputy assistant director of the Secret Service’s Office of Government and Public Affairs, emphasizes that sarcasm detection is only one of many requirements. “It’s so you don’t have to sift through thousands of tweets,” he says. “We want to streamline, to automate the social media monitoring process. Not just for sarcasm.”

“Right now, if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.”

Donovan says that assessing large volumes of messages has been a problem for the Secret Service, which in the past has borrowed the analytics tools of other agencies, like FEMA. “It’s an issue that’s come up before, and we don’t want to waste time sifting through messages.”

“Right now,” Donovan says, “if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.” The agency wants to be able to monitor its own presence on social media, but also track trending topics and influential users.

“Remember the purple tunnel of doom?” Donovan asks, referring to the debacle during the 2009 presidential inauguration when thousands of attendances with purple tickets were caught in the Third Street Tunnel, unable to cross 395 to attend the event. “We weren’t monitoring Twitter that day, so we didn’t know. Since then, we’ve entered social media.”

The Secret Service (@SecretService) joined Twitter in February of 2010. The account has posted only 572 tweets, but is followed by 110,000 users. And it’s not the only governmental agency interested in assessing the tone of other users. Last summer, the BBC reported that the French company Spotter provided such an analytics tool to the British Home Office, the European Union Commission, and Dubai Courts.

For around $1,675 per month, Spotter provides software that “uses a combination of linguistics, semantics and heuristics to create algorithms that generate reports about online reputation.” Determining whether users are genuinely complaining or sincerely threatening a government agency is difficult, but Spotter claims an 80 percent accuracy rate and offers the service in 29 languages.

But how does software accomplish a task that is difficult even for sentient beings? Even the most casual of correspondents has had a text message or email misunderstood by its recipient. Without the aid of eye rolls or shoulder shrugs, textual sarcasm can be difficult to detect. The science of sincerity, or sarcasm if you’re a glass-empty type, is critical for law enforcement agencies, but also profitable for corporations seeking to improve their customer service. Understanding whether a reviewer sarcastically praised your product or honestly questioned your services is critical.

So what used to be the domain of literary critics has become a novel problem for software designers. A 2010 paper by three researchers at Hebrew University presented a sarcasm algorithm with a 77 percent precision rate. Developed through an analysis of 66,000 Amazon product reviews, the algorithm was able to detect the situational irony of titles (“[I] Love the Cover” for a book review and “Where Am I?” for a GPS device) and sarcastic patterns in speech and punctuation (sentence length, multiple exclamation or question marks in sentences, and the number of words with all capital letters among others).

But Amazon reviews have many words. In a 2011 paper, three researchers at Rutgers’ School of Communication & Information were much less successful at identifying sarcasm on Twitter, which is limited to 140-character tweets. Tweet length as well as the lack of context made assessing tone very difficult, for machines but also humans. Even with the assistance of Twitter tics like emoticons and hashtags, the human judges and the machines demonstrated only 70 percent accuracy when distinguishing sarcastic tweets from positives or negative tweets.

But that was 2011, and with companies like Spotter already reporting 80 percent accuracy, the Secret Service will likely find more bids than it needs. Donovan expressed some sarcasm of his own when I asked about how they will measure the actual accuracy of whichever software they purchase. “That’s a great question,” Donovan says, “for the companies that make these claims.”

Casey N. Cep
Casey N. Cep is a writer from the Eastern Shore of Maryland. She has written for the New Republic, the New York Times, the New Yorker, and the Paris Review. Follow her on Twitter @cncep.

More From Casey N. Cep

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

July 24 • 4:00 PM

Overweight Americans Have the Lowest Risk of Premature Death

Why do we use the term “normal weight” when talking about BMI? What’s presented as normal certainly isn’t the norm, and it may not even be what’s most healthy.


July 24 • 2:00 PM

California’s Lax Policing of the Fracking Industry Has Put the Drought-Stricken State in a Terrible Situation

The state’s drought has forced farmers to rely on groundwater, even as aquifers have been intentionally polluted due to exemptions for the oil industry.


July 24 • 12:00 PM

What’s in a Name? The Problem With Washington’s Football Team

A senior advisor to the National Congress of American Indians once threw an embarrassing themed party that involved headdresses. He regrets that costume now, but knows his experience is one many others can relate to.


July 24 • 11:00 AM

How Wildlife Declines Are Leading to Slavery and Terrorism

As wildlife numbers dwindle, wildlife crimes are rising—and that’s fueling a raft of heinous crimes committed against humans.


July 24 • 10:58 AM

How the Supremes Pick Their Cases—and Why Obamacare Is Safe for Now

The opponents of Obamacare who went one for two in circuit court rulings earlier this week are unlikely to see their cases reach the Supreme Court.



July 24 • 9:48 AM

The People Who Are Scared of Dogs

While more people fear snakes or spiders, with dogs everywhere, cynophobia makes everyday public life a constant challenge.


July 24 • 8:00 AM

Newton’s Needle: On Scientific Self-Experimentation

It is all too easy to treat science as a platform that allows the observer to hover over the messiness of life, unobserved and untouched. But by remembering the role of the body in science, perhaps we humanize it as well.


July 24 • 6:00 AM

Commercializing the Counterculture: How the Summer Music Festival Went Mainstream

With painted Volkswagen buses, talk of “free love,” and other reminders of the Woodstock era replaced by advertising and corporate sponsorships, hippie culture may be dying, but a new subculture—a sort of purgatory between hipster and hippie—is on the rise.


July 24 • 5:00 AM

In Praise of Our Short Attention Spans

Maybe there’s a good reason why it seems like there’s been a decline in our our ability to concentrate for a prolonged period of time.


July 24 • 4:00 AM

How Stereotypes Take Shape

New research from Scotland finds they’re an unfortunate product of the way we process and share information.


July 23 • 4:00 PM

Who Doesn’t Like Atheists?

The Pew Research Center asked Americans of varying religious affiliations how they felt about each other.


July 23 • 2:00 PM

We Need to Start Tracking Patient Harm and Medical Mistakes Now

Top patient-safety experts call on Congress to step in and, among other steps, give the Centers for Disease Control and Prevention wider responsibility for measuring medical mistakes.


July 23 • 12:19 PM

How a CEO’s Fiery Battle Speeches Can Shape Ethical Behavior

CEO war speech might inspire ethical decisions internally and unethical ones among competing companies.


July 23 • 12:00 PM

Why Do We Love the ‘Kim Kardashian: Hollywood’ Game?

It’s easy enough to turn yourself into a virtual celebrity, complete with fame and mansions—but it will likely cost you.


July 23 • 11:49 AM

Modern Technology Still Doesn’t Protect Americans From Deadly Landslides

No landslide monitoring or warning systems are being used to protect vulnerable communities.


July 23 • 10:00 AM

Outing the Death-Drug Distributors

Calling all hackers: It’s time to go Assange on capital punishment.


July 23 • 8:00 AM

The Surprising Appeal of Products That Require Effort to Use

New research finds they enable consumers to re-establish a feeling that they’re in control of their lives.



July 23 • 6:00 AM

How the Other Half Lifts: What Your Workout Says About Your Social Class

Why can’t triathletes and weightlifters get along?


July 23 • 5:02 AM

Battle of the Public Intellectuals: Edward Glaeser vs. Richard Florida

On gentrification and housing costs.


July 23 • 4:00 AM

Our Fear of Immigrants

Why did a group of fourth graders rally in support of an undocumented classmate while the citizens of Murrieta, California, tried to stop immigrant children from entering their town?


July 22 • 4:00 PM

Can Meditation Really Slow Aging?

Is there real science in the spiritualism of meditation? Jo Marchant meets a Nobel Prize-winner who thinks so.



July 22 • 2:00 PM

The Alabama Judge Who Refuses to Let Desegregation Orders Go Ignored

A federal judge in Alabama says a local school board has failed to meet legal mandate to integrate.


Follow us


Subscribe Now

How Wildlife Declines Are Leading to Slavery and Terrorism

As wildlife numbers dwindle, wildlife crimes are rising—and that's fueling a raft of heinous crimes committed against humans.

How a CEO’s Fiery Battle Speeches Can Shape Ethical Behavior

CEO war speech might inspire ethical decisions internally and unethical ones among competing companies.

Modern Technology Still Doesn’t Protect Americans From Deadly Landslides

No landslide monitoring or warning systems are being used to protect vulnerable communities.

The Link Between Carbs, Gut Microbes, and Colon Cancer

Reduced carb intake among mice protected them from colon cancer.

The New Weapon Against Disease-Spreading Insects Is Big Data

Computer models that pinpoint the likely locations of mosquitoes and tsetse flies are helping officials target vector control efforts.

The Big One

Today, the United States produces less than two percent of the clothing purchased by Americans. In 1990, it produced nearly 50 percent. July/August 2014

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.