Menus Subscribe Search

In the Classroom

acorn-leaf

(Photo: randihausken/Flickr)

Filling in the Blanks: The Thousands of Volunteers Who Grade Millions of AP Tests

• July 09, 2014 • 10:00 AM

(Photo: randihausken/Flickr)

It’s sort of like summer camp—just for highly educated adults.

Seven years ago, Tom James was de-corned. James didn’t undergo a medical procedure. Rather, the math and astronomy instructor at Chaffey High School in Ontario, California, was where he’s been every June since: in the Kansas City, Missouri Convention Center, grading thousands of Advanced Placement tests. Every first-year grader, or AP Reader, has a small acorn, the logo of the College Board, printed on the corner of his official nametag. To celebrate surviving the first week, a Reader’s table partner rips off the acorn—the official de-corning.

James’s table partner that first year, Mark Kannowski, did the honors. Kannowski is a mathematics professor and the former department chair at DePauw University in Greencastle, Indiana. After sharing the ritual, the two became friends who exchange emails about pedagogy and politics throughout the year. Kannowski, an experienced grader, has been reading AP Exams on and off since 1991. Every summer they meet in Kansas City, James and Kannowski have dinner and try to catch a Royals game.

“It’s a nice environment where egos have been checked at the door. College professors don’t look down on us at all. After we get over the initial shock of their acceptance, we don’t feel intimidated either.”

The Advanced Placement pilot program launched shortly after World War II to help close the widening gap in academic achievement between high school and university. In the 1950s, offering college-level curricula to high school students seemed groundbreaking. But more than 60 years later, it’s often considered an obvious option to improve secondary education, even though it still isn’t available in every school or to every student. (Home-schooled students, for instance, cannot take AP courses, but any student can take any AP exam.)

Much like other nationwide standardized testing—the SAT and ACT being perhaps the most recognizable—Advanced Placement testing occurs annually at high schools all over the United States. Testing follows a year or more of enrollment in an AP course, such as English, chemistry, or calculus, for which students can earn college credit and test out of freshman-level university courses. AP Exams aren’t given a traditional letter grade, but they instead measure, on a one-to-five scale, how prepared and qualified students are for any given subject at a two- or four-year institution of higher education in the U.S. or Canada. And like any test with a written component, real people—for AP Exams, thousands of teachers from all over the country—grade the essays.

BY THE TIME THEY’RE ripping up their nametags, the Exam Readers need a break. “We read for seven straight days,” James says, though he notes the first day is mostly for getting organized, and the last day typically serves as a way to wind down. There are six questions per exam, and on average, he says, a Reader will grade 2,900 questions. Divide by seven—or five—and you’re looking at well over 400 to 500 questions a day.

Multiply that out to include every AP Reader, and you end up with some staggering (official) statistics following last year’s testing. More than 2.2 million students took an AP Exam in 2013. Not all of those required hand grading. But according to the College Board, over just three weeks that summer, 11,497 AP teachers and professors from two- and four-year colleges spent 643,832 total hours reading and scoring 17.8 million student responses from over 3.9 million AP exams.

That’s a lot of bad penmanship to decipher.

The time and resources spent seems inconceivable when you look at the raw data. As additional tests have been added, says Katherine Levin of the College Board, “the number of Readers has nearly doubled over the last decade.” Since AP testing began in 1956, half of the exams have always been put in the hands of teachers from around the country. Computers grade the other half.

More than 25 subjects are graded by hand each year, and organizers separate readers into several cities and convention centers according to their subject matter. Twenty-two teachers sit around a table, reading packets that contain 25 answers to each of the six test questions. If the person is a new Reader, a Table Reader will look at his or her first completed folder. The next level up, the Question Leader (Kannowski is currently one), is responsible for making an initial presentation to the Readers and Table Leaders, and overseeing specific problems and writing reports about issues that come up. Question and Table Leaders get invited to take leadership positions based on their experience as Readers—like a miniature tenure system of sorts.

The grading process becomes collaborative—and not just because it’s set up to be that way. “If [several Readers at our table] don’t get same scores, we’ll confer and discuss. Sometimes I’ll change my mind, but sometimes the Table Leader will change her mind,” James says, pausing. “It’s really a neat thing. It helps me get better attuned to the flow of thinking, and helps make sure I’m interpreting things correctly. It’s neat when someone else says, ‘I agree with you.’” This is especially exciting for James, who notes the graders are a “wonderfully diverse group in every way you can think of.” He says the only homogeneity in the calculus grading group is that everyone is a math teacher.

The subjects tend to be as diverse as the instructors tasked with grading them all. Music theory, macro- and microeconomics, and languages including French, German, Italian, and half of the Spanish tests are all graded in Cincinnati, Ohio, at the Duke Energy Convention Center during the first week in June. (The other 50 percent of the Spanish tests are graded during the third week of the month.) Japanese, Chinese, and Latin, along with subjects including studio art and art history, are graded during the second week of June in Salt Lake City, Utah, at the Salt Lake Palace Convention Center.

“[Grading] is a supremely nerdy thing to do,” James admits with a chuckle. “But it’s so interesting and satisfying.” The College Board provides travel, lodging, and meals, along with an honorarium of $1,639, for every Reader. And while there is an application process to become a Reader, most who apply do get accepted.

“One thing that’s impressed me every year is the relationships between teachers and professors,” he says. “It’s a nice environment where egos have been checked at the door. College professors don’t look down on us [high school teachers] at all. After we get over the initial shock of their acceptance, we don’t feel intimidated either.”

From his perspective at the university level, Kannowski says that grading helps him understand the aptitude of rising freshmen. He also suggests that the AP program can fill the gap between secondary and higher education. “I think that, as the cost of college and university increases, [AP testing] is another important way for students to be educated.”

As the American education system has rapidly changed over the past 20 years—and seems set for even more upheaval—James considers teaching students to flex their critical thinking skills to be, well, critical. “No one’s going to have a test bubbling in Scantron sheets,” he says. “We simply don’t live in a multiple-choice world.” Learning to explain choices and defend decisions are skills that must be cultivated—even if it demands all the resources required to mobilize an army of essay-reading teachers to Midwest conference centers every June.

Brittany Shoot
Brittany Shoot is a journalist reporting on people, places, and products. She is also the managing editor of The Magazine. Follow her on Twitter @brittanyshoot.

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

July 29 • 6:00 AM

What Are the Benefits of Locking Yourself in a Tank and Floating in Room-Temperature Saltwater?

After three sessions in an isolation tank, the answer’s still not quite clear.


July 29 • 4:00 AM

Harry Potter and the Battle Against Bigotry

Kids who identify with the hero of J.K. Rowling’s popular fantasy novels hold more open-minded attitudes toward immigrants and gays.


July 29 • 2:00 AM

Geographic Scale and Talent Migration: Washington, D.C.’s New Silver Line

Around the country, suburbs are fighting with the urban core over jobs and employees.


July 28 • 4:00 PM

Border Fences Make Unequal Neighbors and Enforce Social Inequality

What would it look like if you combined Israel, the West Bank, and Gaza, demographically speaking? What about the United States and Guatemala?


July 28 • 2:00 PM

Are Patient Privacy Laws Being Misused to Protect Medical Centers?

A 1996 law known as HIPAA has been cited to scold a mom taking a picture of her son in a hospital, to keep information away from police investigating a possible rape at a nursing home, and to threaten VA whistleblowers.


July 28 • 12:00 PM

Does Internet Addiction Excuse the Death of an Infant?

In Love Child, documentary filmmaker Valerie Veatch explores how virtual worlds encourage us to erase the boundary between digital and real, no matter the consequences.


July 28 • 11:11 AM

NASA Could Build Entire Spacecrafts in Space Using 3-D Printers

This year NASA will experiment with 3-D printing small objects in space. That could mark the beginning of a gravity-free manufacturing revolution.


July 28 • 10:00 AM

Hell Isn’t for Real

You may have seen pictures of the massive crater in Siberia. It unfortunately—or fortunately—does not lead to the netherworld.


July 28 • 8:00 AM

Why Isn’t Obama More Popular?

It takes a while for people to notice that things are going well, particularly when they’ve been bad for so long.


July 28 • 7:45 AM

The Most Popular Ways to Share Good and Bad Personal News

Researchers rank the popularity of all of the different methods we have for telling people about our lives, from Facebook to face-to-face.


July 28 • 6:00 AM

Hams Without Ends and Cats Tied to Trees: How We Create Traditions With Dubious Origins

Does it really matter if the reason for why you give money to newlyweds is based on a skewed version of a story your parents once told you?


July 28 • 4:00 AM

A Belief in ‘Oneness’ Is Equated With Pro-Environment Behavior

New research finds a link between concern for the environment and belief in the concept of universal interconnectedness.


July 25 • 4:00 PM

Flying Blind: The View From 30,000 Feet Puts Everything in Perspective

Next time you find yourself in an airplane, consider keeping your phone turned off and the window open.


July 25 • 2:00 PM

Trophy Scarves: Race, Gender, and the Woman-as-Prop Trope

Social inequality unapologetically laid bare.


July 25 • 1:51 PM

Confusing Population Change With Migration

A lot of population change is baked into a region from migration that happened decades ago.


July 25 • 1:37 PM

Do Not Tell Your Kids That Eating Vegetables Will Make Them Stronger

Instead, hand them over in silence. Or, market them as the most delicious snack known to mankind.



July 25 • 11:07 AM

The West’s Groundwater Is Being Sucked Dry

Scientists were stunned to discover just how much groundwater has been lost from beneath the Colorado River over the past 10 years.


July 25 • 10:00 AM

Shelf Help: New Book Reviews in 100 Words or Less

What you need to know about Bad Feminist, XL Love, and The Birth of Korean Cool.



July 25 • 8:00 AM

The Consequences of Curing Childhood Cancer

The majority of American children with cancer will be cured, but it may leave them unable to have children of their own. Should preserving fertility in cancer survivors be a research priority?


July 25 • 6:00 AM

Men Find Caring, Understanding Responses Sexy. Women, Not So Much

For women looking to attract a man, there are advantages to being a caring conversationalist. But new research finds it doesn’t work the other way around.


July 25 • 4:00 AM

Arizona’s Double-Talk on Execution and Torture

The state is certain that Joseph Wood’s death was totally constitutional. But they’re looking into it.


July 24 • 4:00 PM

Overweight Americans Have the Lowest Risk of Premature Death

Why do we use the term “normal weight” when talking about BMI? What’s presented as normal certainly isn’t the norm, and it may not even be what’s most healthy.


July 24 • 2:00 PM

California’s Lax Policing of the Fracking Industry Has Put the Drought-Stricken State in a Terrible Situation

The state’s drought has forced farmers to rely on groundwater, even as aquifers have been intentionally polluted due to exemptions for the oil industry.


Follow us


Subscribe Now

NASA Could Build Entire Spacecrafts in Space Using 3-D Printers

This year NASA will experiment with 3-D printing small objects in space. That could mark the beginning of a gravity-free manufacturing revolution.

The Most Popular Ways to Share Good and Bad Personal News

Researchers rank the popularity of all of the different methods we have for telling people about our lives, from Facebook to face-to-face.

Do Not Tell Your Kids That Eating Vegetables Will Make Them Stronger

Instead, hand them over in silence. Or, market them as the most delicious snack known to mankind.

The West’s Groundwater Is Being Sucked Dry

Scientists were stunned to discover just how much groundwater has been lost from beneath the Colorado River over the past 10 years.

How Wildlife Declines Are Leading to Slavery and Terrorism

As wildlife numbers dwindle, wildlife crimes are rising—and that's fueling a raft of heinous crimes committed against humans.

The Big One

One in two full-time American fast-food workers' families are enrolled in public assistance programs, at a cost of $7 billion per year. July/August 2014

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.