Menus Subscribe Search
babbage-engine

The Babbage Analytical Engine, for which the first algorithm was designed. (PHOTO: GENI/WIKIMEDIA COMMONS)

5 Hidden Algorithms That Rule Your World

• November 12, 2013 • 8:00 AM

The Babbage Analytical Engine, for which the first algorithm was designed. (PHOTO: GENI/WIKIMEDIA COMMONS)

From the NSA to OKCupid.

If the 1990s were dominated by the rise of the Internet and the 2000s social media behemoths, perhaps the ’10s are the decade of the algorithm. The word is a kind of cypher for whatever is happening in the world of technology at the moment. A brief survey of recent stories on the subject includes a bridge designed by algorithm, a traffic-reducing algorithm, and a match-making algorithm.

But what is an algorithm, exactly?

An algorithm is a method of processing information, a step-by-step procedure that turns a certain set of data into something else—perhaps a smaller, filtered set of data, an outcome that’s more useful than the original for the user of the algorithm. It’s a process-driven equation, one that’s difficult to define except in vague terms.

The word itself has ancient roots. Muhammad ibn Musa al-Khwarizmi, a mathematician in Baghdad in the 9th century, is known as the creator of algebra, which stems from the word al-jabr, meaning completion, in the context of an equation. In Latin translation, he became Algorismus, which was synonymous with the decimal number system, which the mathematician introduced to the Western world. Algorism was the practice of doing arithmetic with those digits.

Compounding what you’ve previously searched with what other users search for every day, Google’s algorithms can predict what you’re looking for with a frightening degree of accuracy.

Algorithm’s modern usage came to the fore with the evolution of computers. Ada Lovelace was a British mathematician in the mid-19th century who worked with Charles Babbage’s analytical engines—the first computers—and is credited with creating the first computer program, an algorithm designed to calculate Bernoulli numbers, critical to the number theory field of mathematics.

Lovelace imagined that computers might one day do much more than determine numbers, however important. If the “science of harmony” could be calculated, she once wrote, “the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.”

But she couldn’t have known the degree to which algorithms would metastasize and come to take over our lives in the 21st century. Below, I introduce five modern algorithms that we should all be aware of, since they alter our technological landscape in much more insidious ways than the Babbage Engine ever did.

1. The NSA’s Surveillance Algorithm

The NSA’s surveillance programs work, or so they claim, by collecting metadata: not the content of your phone calls and emails, but their length, who they connect you to, and when they happen. This mass of metadata is filtered by the agency’s algorithms, which trawl through the collection for triggers that give them reason for further investigation, AKA requisitioning the content of the communications. These triggers could be certain keywords—“offensive information warfare,” “illuminati,” and “computer terrorism” to name a few—or the identities of the people you’re contacting.

In one of the few concrete examples of successful metadata surveillance, FBI Deputy Director Sean Joyce stated in June that a phone number in San Diego was found to be making calls to the terrorist group al-Shabab in Somalia. The agency traced the number to one man, and further surveillance connected him to three others who were conspiring to send money to the terrorist group. All four were arrested in 2010 and convicted early this year.

Of course, now that details about the algorithm have spread, there’s a good chance it will be less effective. Vice recently launched an NSA spam generator that spits out sentences full of trigger words for would-be surveillance disruptors to insert into their email and confuse the system.

2. Encryption Algorithms

Another way to combat surveillance is to confuse the algorithms—turning their own capabilities against them. Encryption algorithms encode your data, using a secret formula to turn it into an unreadable mass that entities like the NSA can’t process. That’s how post-PRISM services like Silent Circle and Least Authority protect their users’ phone calls and messages from snooping. But encryption algorithms have a very long history.

According to the Roman historian Suetonius, Julius Caesar used the “Caesar cipher” to encode his private correspondence. The algorithm worked by shifting each letter of the alphabet back several places—for example, every D would become an A, every J would become a G, and so on. The message could only be unencrypted if the receiver knew what the shift was. This is a simplistic, easily-broken algorithm, but the German Enigma machine, which was used to encode military texts after World War I, took almost two decades to crack. These days, the NSA just goes straight to the source: It spies on Angela Merkel’s cell phone.

3. Google Search

If you’ve ever had Google fill in the rest of your search text like a mind-reader, you’ve also experienced an algorithm at work. Compounding what you’ve previously searched with what other users search for every day, Google’s algorithms can predict what you’re looking for with a frightening degree of accuracy.

In fact, Google’s search algorithm, which is also referred to as PageRank, is one of the most powerful, secretive algorithms in the world of technology. In a visual explainer of how search works, Google notes that their algorithms “get to work looking for clues to better understand what you mean” by your search terms rather than just their literal content. They then use their formula to determine how relevant a certain page is to what you’re looking for, examining aspects like how often the site is updated and how well it’s connected to the rest of the Internet, putting the most useful links at the top of the search list.

Search-engine optimization (SEO) experts work to crack the code of this algorithm and get their clients’ sites to the top of the page. That’s a tough job, because the algorithm is always evolving, to the tune of 500 to 600 changes a year, to throw off cheaters and better serve searchers.

4. High-Frequency Trading Algorithms

In the heyday of Wall Street, traders shouted across exchange floors to buy and sell stocks. These days, it’s a whole lot quieter—around 50 percent of trading is done by firms specializing in “high-frequency trading” (HFT), which refers to rapid-fire share buys and sells carried out by computer programs rather than humans. Hedge funds that specialize in HFT develop proprietary algorithms to determine what they buy and sell and when, then the programs carry out those instructions in milli- or microseconds. These incredibly fast trades attempt to capitalize on volatile markets, buying before rises and selling before falls, however miniscule. The technology began its meteoric rise in the early ‘90s and became pervasive by 2010, but it’s not without its problems.

“Left unsupervised, algorithms can and will do strange things,” writes Christopher Steiner in his book Automate This: How Algorithms Came to Rule Our World. In 2012, a “rogue algorithm” belonging to the New York firm Knight Capital glitched out and went on an ill-advised 45-minute trading spree that resulted in a loss of $440 million before it was quickly stopped, Steiner writes. The Flash Crash of 2010, in which the Dow Jones Industrial Average lost 1,000 points in a few minutes only to recover the value 20 minutes later, was exacerbated by high-frequency trading. When the algorithms noticed the downswing in the market, firms depending on the programs sold aggressively to protect their investors. In fact, the strategy might be on its way out—France and Italy have both instituted taxes on light-speed trading, a law that the U.S. is considering as well.

5. OKCupid

Algorithms influence even our intimate relationships. The now-iconic surveys of OKCupid, the question-and-answer sessions the dating site spews out to help match its users, provide the data for the site’s compatibility algorithms. In order to calculate a match percentage between you and another person, the algorithm will factor in the results of the survey questions’ three aspects: what your answer is, what answer you’d like the other person to give, and how important the question is to you. In the computer program, those abstract values are assigned numbers, which are collated into a “match percentage.”

The company explains that if a person chooses the exact answer that you specify you want, they receive all of the matching points accorded to the level of importance you selected. So if you wanted a match to answer “How messy are you?” at the middle level of “Average,” and said that answer is “very important,” and the match chose that answer, the match would receive 50 out of 50 possible points. If they answered the question “Have you ever cheated in a relationship?” “Yes” but you specified a no answer at a very important level, they would receive zero out of 50 points.

As the question results mount up, those fractions are calculated into a final match percentage. For our hypothetical match, 50/50 plus 0/50 would amount to 50 out of 100 total points, or a miserable 50 percent compatibility rating. Ouch. No way an algorithm would go for that.

Algorithms might be pure math, but in all of these cases they are created and controlled by fallible human beings. These equations and data filters might be quietly influencing our lives, but that doesn’t mean we’re controlled by robot overlords: Imperfect humans create imperfect algorithms. In other words, you never know about that 50-percent match.

Kyle Chayka
Kyle Chayka is a freelance technology and culture writer living in Brooklyn. Follow him on Twitter @chaykak.

More From Kyle Chayka

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

July 29 • 4:00 PM

Are Children Seeking Refuge Turning More Americans Against Undocumented Immigrants?

A look at Pew Research Center survey data collected in February and July of this year.


July 29 • 2:00 PM

Under Water: The EPA’s Ongoing Struggle to Combat Pollution

Frustration and inaction color efforts to enforce the Clean Water Act.


July 29 • 12:40 PM

America’s Streams Are Awash With Pesticides Banned in Europe

You may have never heard of clothianidin, but it’s probably in your local river.


July 29 • 12:00 PM

Mining Your Genetic Data for Profit: The Dark Side of Biobanking

One woman’s personal story raises deep questions about the stark limits of current controls in a nascent industry at the very edge of the frontier of humans and technology.


July 29 • 11:23 AM

Where Should You Go to College?


July 29 • 10:29 AM

How Textbooks Have Changed the Face of War

War is more personal, less glorious, and more hellish in modern textbooks than in the past. But there’s still room for improvement.


July 29 • 10:00 AM

The Monolingual American: Why Are Those Outside of the U.S. Encouraging It?

If you are an American trying to learn German in a large German town or city, you will mostly hear English in return, even when you give sprechen your best shot.


July 29 • 8:00 AM

The Elusive Link Between Casinos and Crime

With a study of the impact of Philadelphia’s SugarHouse Casino, a heated debate gets fresh ammunition.


July 29 • 6:00 AM

What Are the Benefits of Locking Yourself in a Tank and Floating in Room-Temperature Saltwater?

After three sessions in an isolation tank, the answer’s still not quite clear.


July 29 • 4:00 AM

Harry Potter and the Battle Against Bigotry

Kids who identify with the hero of J.K. Rowling’s popular fantasy novels hold more open-minded attitudes toward immigrants and gays.


July 29 • 2:00 AM

Geographic Scale and Talent Migration: Washington, D.C.’s New Silver Line

Around the country, suburbs are fighting with the urban core over jobs and employees.


July 28 • 4:00 PM

Border Fences Make Unequal Neighbors and Enforce Social Inequality

What would it look like if you combined Israel, the West Bank, and Gaza, demographically speaking? What about the United States and Guatemala?


July 28 • 2:00 PM

Are Patient Privacy Laws Being Misused to Protect Medical Centers?

A 1996 law known as HIPAA has been cited to scold a mom taking a picture of her son in a hospital, to keep information away from police investigating a possible rape at a nursing home, and to threaten VA whistleblowers.


July 28 • 12:00 PM

Does Internet Addiction Excuse the Death of an Infant?

In Love Child, documentary filmmaker Valerie Veatch explores how virtual worlds encourage us to erase the boundary between digital and real, no matter the consequences.


July 28 • 11:11 AM

NASA Could Build Entire Spacecrafts in Space Using 3-D Printers

This year NASA will experiment with 3-D printing small objects in space. That could mark the beginning of a gravity-free manufacturing revolution.


July 28 • 10:00 AM

Hell Isn’t for Real

You may have seen pictures of the massive crater in Siberia. It unfortunately—or fortunately—does not lead to the netherworld.


July 28 • 8:00 AM

Why Isn’t Obama More Popular?

It takes a while for people to notice that things are going well, particularly when they’ve been bad for so long.


July 28 • 7:45 AM

The Most Popular Ways to Share Good and Bad Personal News

Researchers rank the popularity of all of the different methods we have for telling people about our lives, from Facebook to face-to-face.


July 28 • 6:00 AM

Hams Without Ends and Cats Tied to Trees: How We Create Traditions With Dubious Origins

Does it really matter if the reason for why you give money to newlyweds is based on a skewed version of a story your parents once told you?


July 28 • 4:00 AM

A Belief in ‘Oneness’ Is Equated With Pro-Environment Behavior

New research finds a link between concern for the environment and belief in the concept of universal interconnectedness.


July 25 • 4:00 PM

Flying Blind: The View From 30,000 Feet Puts Everything in Perspective

Next time you find yourself in an airplane, consider keeping your phone turned off and the window open.


July 25 • 2:00 PM

Trophy Scarves: Race, Gender, and the Woman-as-Prop Trope

Social inequality unapologetically laid bare.


July 25 • 1:51 PM

Confusing Population Change With Migration

A lot of population change is baked into a region from migration that happened decades ago.


July 25 • 1:37 PM

Do Not Tell Your Kids That Eating Vegetables Will Make Them Stronger

Instead, hand them over in silence. Or, market them as the most delicious snack known to mankind.



Follow us


Subscribe Now

America’s Streams Are Awash With Pesticides Banned in Europe

You may have never heard of clothianidin, but it's probably in your local river.

How Textbooks Have Changed the Face of War

War is more personal, less glorious, and more hellish in modern textbooks than in the past. But there’s still room for improvement.

NASA Could Build Entire Spacecrafts in Space Using 3-D Printers

This year NASA will experiment with 3-D printing small objects in space. That could mark the beginning of a gravity-free manufacturing revolution.

The Most Popular Ways to Share Good and Bad Personal News

Researchers rank the popularity of all of the different methods we have for telling people about our lives, from Facebook to face-to-face.

Do Not Tell Your Kids That Eating Vegetables Will Make Them Stronger

Instead, hand them over in silence. Or, market them as the most delicious snack known to mankind.

The Big One

One in two full-time American fast-food workers' families are enrolled in public assistance programs, at a cost of $7 billion per year. July/August 2014

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.