Menus Subscribe Search

The World Wide Web

secret-service

(Photo: David Stuart Productions/Shutterstock)

Can We Really Detect Sarcasm With a Machine?

• June 17, 2014 • 10:00 AM

(Photo: David Stuart Productions/Shutterstock)

What was once the domain of literary critics has now become the world of the Secret Service.

Eyes rolled a couple of weeks ago when the Secret Service posted a work order for new social media analytics software. Not because the law enforcement agency tasked with protecting the president and other federal officials lacks a social media presence, but because among the 22 functionality requirements for the software was the “ability to detect sarcasm and false positives.”

Edwin M. Donovan, deputy assistant director of the Secret Service’s Office of Government and Public Affairs, emphasizes that sarcasm detection is only one of many requirements. “It’s so you don’t have to sift through thousands of tweets,” he says. “We want to streamline, to automate the social media monitoring process. Not just for sarcasm.”

“Right now, if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.”

Donovan says that assessing large volumes of messages has been a problem for the Secret Service, which in the past has borrowed the analytics tools of other agencies, like FEMA. “It’s an issue that’s come up before, and we don’t want to waste time sifting through messages.”

“Right now,” Donovan says, “if there were suddenly 200 tweets about something on D Street near the Capitol, we’d want to be able to synthesize that.” The agency wants to be able to monitor its own presence on social media, but also track trending topics and influential users.

“Remember the purple tunnel of doom?” Donovan asks, referring to the debacle during the 2009 presidential inauguration when thousands of attendances with purple tickets were caught in the Third Street Tunnel, unable to cross 395 to attend the event. “We weren’t monitoring Twitter that day, so we didn’t know. Since then, we’ve entered social media.”

The Secret Service (@SecretService) joined Twitter in February of 2010. The account has posted only 572 tweets, but is followed by 110,000 users. And it’s not the only governmental agency interested in assessing the tone of other users. Last summer, the BBC reported that the French company Spotter provided such an analytics tool to the British Home Office, the European Union Commission, and Dubai Courts.

For around $1,675 per month, Spotter provides software that “uses a combination of linguistics, semantics and heuristics to create algorithms that generate reports about online reputation.” Determining whether users are genuinely complaining or sincerely threatening a government agency is difficult, but Spotter claims an 80 percent accuracy rate and offers the service in 29 languages.

But how does software accomplish a task that is difficult even for sentient beings? Even the most casual of correspondents has had a text message or email misunderstood by its recipient. Without the aid of eye rolls or shoulder shrugs, textual sarcasm can be difficult to detect. The science of sincerity, or sarcasm if you’re a glass-empty type, is critical for law enforcement agencies, but also profitable for corporations seeking to improve their customer service. Understanding whether a reviewer sarcastically praised your product or honestly questioned your services is critical.

So what used to be the domain of literary critics has become a novel problem for software designers. A 2010 paper by three researchers at Hebrew University presented a sarcasm algorithm with a 77 percent precision rate. Developed through an analysis of 66,000 Amazon product reviews, the algorithm was able to detect the situational irony of titles (“[I] Love the Cover” for a book review and “Where Am I?” for a GPS device) and sarcastic patterns in speech and punctuation (sentence length, multiple exclamation or question marks in sentences, and the number of words with all capital letters among others).

But Amazon reviews have many words. In a 2011 paper, three researchers at Rutgers’ School of Communication & Information were much less successful at identifying sarcasm on Twitter, which is limited to 140-character tweets. Tweet length as well as the lack of context made assessing tone very difficult, for machines but also humans. Even with the assistance of Twitter tics like emoticons and hashtags, the human judges and the machines demonstrated only 70 percent accuracy when distinguishing sarcastic tweets from positives or negative tweets.

But that was 2011, and with companies like Spotter already reporting 80 percent accuracy, the Secret Service will likely find more bids than it needs. Donovan expressed some sarcasm of his own when I asked about how they will measure the actual accuracy of whichever software they purchase. “That’s a great question,” Donovan says, “for the companies that make these claims.”

Casey N. Cep
Casey N. Cep is a writer from the Eastern Shore of Maryland. She has written for the New Republic, the New York Times, the New Yorker, and the Paris Review. Follow her on Twitter @cncep.

More From Casey N. Cep

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

July 30 • 8:00 AM

How to Make a Convincing Sci-Fi Movie on a Tight Budget

Coherence is a good movie, and its initial shoot cost about the same amount of money as a used Prius.


July 30 • 6:00 AM

Are You Really as Happy as You Say You Are?

Researchers find a universal positivity bias in the way we talk, tweet, and write.


July 30 • 4:00 AM

The Declining Wage Gap for Gay Men

New research finds gay men in America are rapidly catching up with straight married men in terms of wages.


July 30 • 2:00 AM

LeBron James Migration: Big Chef Seeking Small Pond

The King’s return to Cleveland is a symbol for the dramatic shift in domestic as well as international migration.


July 29 • 4:00 PM

Are Children Seeking Refuge Turning More Americans Against Undocumented Immigrants?

A look at Pew Research Center survey data collected in February and July of this year.


July 29 • 2:00 PM

Under Water: The EPA’s Ongoing Struggle to Combat Pollution

Frustration and inaction color efforts to enforce the Clean Water Act.


July 29 • 12:40 PM

America’s Streams Are Awash With Pesticides Banned in Europe

You may have never heard of clothianidin, but it’s probably in your local river.


July 29 • 12:00 PM

Mining Your Genetic Data for Profit: The Dark Side of Biobanking

One woman’s personal story raises deep questions about the stark limits of current controls in a nascent industry at the very edge of the frontier of humans and technology.


July 29 • 11:23 AM

Where Should You Go to College?


July 29 • 10:29 AM

How Textbooks Have Changed the Face of War

War is more personal, less glorious, and more hellish in modern textbooks than in the past. But there’s still room for improvement.


July 29 • 10:00 AM

The Monolingual American: Why Are Those Outside of the U.S. Encouraging It?

If you are an American trying to learn German in a large German town or city, you will mostly hear English in return, even when you give sprechen your best shot.


July 29 • 8:00 AM

The Elusive Link Between Casinos and Crime

With a study of the impact of Philadelphia’s SugarHouse Casino, a heated debate gets fresh ammunition.


July 29 • 6:00 AM

What Are the Benefits of Locking Yourself in a Tank and Floating in Room-Temperature Saltwater?

After three sessions in an isolation tank, the answer’s still not quite clear.


July 29 • 4:00 AM

Harry Potter and the Battle Against Bigotry

Kids who identify with the hero of J.K. Rowling’s popular fantasy novels hold more open-minded attitudes toward immigrants and gays.


July 29 • 2:00 AM

Geographic Scale and Talent Migration: Washington, D.C.’s New Silver Line

Around the country, suburbs are fighting with the urban core over jobs and employees.


July 28 • 4:00 PM

Border Fences Make Unequal Neighbors and Enforce Social Inequality

What would it look like if you combined Israel, the West Bank, and Gaza, demographically speaking? What about the United States and Guatemala?


July 28 • 2:00 PM

Are Patient Privacy Laws Being Misused to Protect Medical Centers?

A 1996 law known as HIPAA has been cited to scold a mom taking a picture of her son in a hospital, to keep information away from police investigating a possible rape at a nursing home, and to threaten VA whistleblowers.


July 28 • 12:00 PM

Does Internet Addiction Excuse the Death of an Infant?

In Love Child, documentary filmmaker Valerie Veatch explores how virtual worlds encourage us to erase the boundary between digital and real, no matter the consequences.


July 28 • 11:11 AM

NASA Could Build Entire Spacecrafts in Space Using 3-D Printers

This year NASA will experiment with 3-D printing small objects in space. That could mark the beginning of a gravity-free manufacturing revolution.


July 28 • 10:00 AM

Hell Isn’t for Real

You may have seen pictures of the massive crater in Siberia. It unfortunately—or fortunately—does not lead to the netherworld.


July 28 • 8:00 AM

Why Isn’t Obama More Popular?

It takes a while for people to notice that things are going well, particularly when they’ve been bad for so long.


July 28 • 7:45 AM

The Most Popular Ways to Share Good and Bad Personal News

Researchers rank the popularity of all of the different methods we have for telling people about our lives, from Facebook to face-to-face.


July 28 • 6:00 AM

Hams Without Ends and Cats Tied to Trees: How We Create Traditions With Dubious Origins

Does it really matter if the reason for why you give money to newlyweds is based on a skewed version of a story your parents once told you?


July 28 • 4:00 AM

A Belief in ‘Oneness’ Is Equated With Pro-Environment Behavior

New research finds a link between concern for the environment and belief in the concept of universal interconnectedness.


July 25 • 4:00 PM

Flying Blind: The View From 30,000 Feet Puts Everything in Perspective

Next time you find yourself in an airplane, consider keeping your phone turned off and the window open.


Follow us


Subscribe Now

America’s Streams Are Awash With Pesticides Banned in Europe

You may have never heard of clothianidin, but it's probably in your local river.

How Textbooks Have Changed the Face of War

War is more personal, less glorious, and more hellish in modern textbooks than in the past. But there’s still room for improvement.

NASA Could Build Entire Spacecrafts in Space Using 3-D Printers

This year NASA will experiment with 3-D printing small objects in space. That could mark the beginning of a gravity-free manufacturing revolution.

The Most Popular Ways to Share Good and Bad Personal News

Researchers rank the popularity of all of the different methods we have for telling people about our lives, from Facebook to face-to-face.

Do Not Tell Your Kids That Eating Vegetables Will Make Them Stronger

Instead, hand them over in silence. Or, market them as the most delicious snack known to mankind.

The Big One

One in two full-time American fast-food workers' families are enrolled in public assistance programs, at a cost of $7 billion per year. July/August 2014

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.