Menus Subscribe Search

Culture Essays

column-culture-movie-1

Seamless Cinema: Speed Racer (2008) had a rough time at the box office, but some of its editing tricks may anticipate the movie language of tomorrow. (Photo: Warner Bros. Pictures)

How the Movies of Tomorrow Will Play With Your Mind

• April 29, 2014 • 8:00 AM

Seamless Cinema: Speed Racer (2008) had a rough time at the box office, but some of its editing tricks may anticipate the movie language of tomorrow. (Photo: Warner Bros. Pictures)

A neuroscientist in California is dreaming up a new visual vocabulary for cinema that could revolutionize the filmmaking business.

Since the dawn of cinema, the cut has been one of the most powerful tools in a director’s kit. If we see a man walk through a door and turn his head to the right, and the scene immediately cuts to an image of an apple on a side table, our brain fills in the gap, and we understand that this man is looking at the apple.

That’s because the brain has a natural propensity for smoothing over interruptions of stimuli. Whenever we blink, our eyes close for up to half a second, but we don’t notice the breaks. We also make rapid eye movements called saccades several times a second as we adjust to a constantly shifting environment, and we lose access to visual information until the eye movement settles down. This may why we generally don’t notice cuts in movies—they work like saccades.

But neuroscientist Sergei Gepshtein dreams of a new visual vocabulary for cinema—one that relies much less on the cut, or perhaps even eliminates the cut altogether. “The film industry rests on a narrow selection of possibilities that got discovered early on and then got canonized by the force of inertia and entrenched by filmmaking technology and habit,” he says.

The potential applications of Gepshtein’s research range widely, from theater and video games to highway signage and building design.

Gepshtein sees some of the most disagreeable traits of entrenched movie technology in today’s blockbuster action movies. In these films, shots last only seconds, and there are regular barrages of rapid-fire cuts. Think Transformers, Battleship, the Bourne trilogy, or Pacific Rim. As Scott Derrickson, director of recent thrillers like Sinister and The Day the Earth Stood Still, laments, “The story is happening to you, but you are not interacting with the story.”

But Gepshtein thinks he can offer an alternative to this trend—and it doesn’t necessarily involve long takes in the style of directors like Alfonso Cuarón, who recently snagged a directing Oscar for Gravity. Instead, it involves harnessing the modern science of vision.

In December, I paid a visit to Gepshtein at his workplace, the Systems Neurobiology Laboratories of the Salk Institute in La Jolla, California, its sleek white facade optimally designed to provide stunning ocean views from nearly every angle. Gepshtein, a slim man with a neatly trimmed beard and wire-frame glasses, met me at the front desk, wearing jeans, a blazer, and a natty beret, which made him look a bit like an auteur director himself. We made our way to his tiny office (space at Salk is at a premium), which was filled with piles of papers and video monitors of every conceivable shape and size.

Over the course of the next few hours, Gepshtein walked me through the historical background to his current research in painstaking detail. Context is everything, and for Gepshtein, everything is context. His conversation was peppered with references to Salvador Dalí, Venetian architecture, Gestalt psychology, optical illusions, philosophy, the novels of Stanislaw Lem and Italo Calvino, and films.

Gepshtein expressed frustration with what he sees as a complacency among filmmakers when it comes to cinematic vocabulary. “They don’t feel handicapped in any way,” he told me. “They don’t know that they sit in the corner of a parameter space.”

By “parameter,” Gepshtein means the parameters of human perception, the focus of his work. Technically, Gepshtein is a vision scientist; he studies what we see. Increasingly, however, he has become fascinated by the ways in which we don’t see. At the heart of his work is something called the window of visibility. This might be defined as the range of what’s in front of our eyes that our brains can actually perceive. Vision is selective: The human eye receives far more visual stimulus than the brain can register.

column-culture-movie-2Lend Me Your Mind: Sergei Gepshtein (left), pictured with his colleague Thomas D. Albright, has big plans for movies. (Photo: Salk Institute for Biological Studies)

As Gepshtein explains, movements that are too fast or too slow won’t make it past the brain’s filter. We know the hour hand of a clock moves, but it moves too slowly for us to perceive any motion with our eyes. We also know that each individual spoke of a spinning wheel is sending information to our eyes, but all we see is a blur.

To determine the dimensions of my own window of visibility, I entered a room even smaller than Gepshtein’s office. There, with the lights turned out, I sat at a desk and peered at a computer monitor on which an image composed of alternating light and dark bands—a tool known as a luminance grating, which dates back to vision research in the 1960s—scrolled past my field of vision, constantly varying in speed, frequency, and degree of contrast.

My instructions were to press one button if the scrolling image seemed to be moving upward and another if it seemed to be moving downward. It started easily, but I soon found myself unable to tell in what direction the gradient was moving. Sometimes it was because the lines had grown too faint; other times because they were moving too quickly or too slowly. It was frustrating—who likes to fail?—but illuminating.

Gepshtein and other scientists have compiled the data from tests like the one I took, and the results are remarkably clear. There is only slight variation in the limits of the window of visibility from person to person. The core data is so consistent that it can be thought of as a law of perception.

Determining the boundaries of the window of visibility has been an exciting leap forward in understanding the limits of visual perception. But to make this knowledge useful to filmmakers, Gepshtein has been enhancing it with other findings from vision science. If the window of visibility is like the outline of a continent on a map, the other findings are like additional geographical information—the location of mountains, rivers, cities. The most important of them have to do with how our brain organizes the things that fall within our window of visibility. To see an example, consider the crisscrossing lines below:

movies-1

Objectively, they are just lines, and they fall easily into your window of visibility. But your brain has organized them into a cube shape. What’s more, your brain has picked one of two ways of looking at the cube. If it has chosen to see it as if you’re looking from above, you’ll effectively see it with the following orientation:

movies-2

But if your brain has chosen to see it as if you’re looking from below, you’ll effectively see it with this orientation:

movies-3

Either orientation can be discerned in the original image, but the brain cannot see both at once. It will pick one. That is an example of perceptual organization, something we all engage in without realizing it.

If the ability to take advantage of such quirks of visual perception could be incorporated into the standard filmmaking software tools, it would put the power of perceptual organization into the hands of the director. In principle, filmmakers could manipulate individual elements onscreen to bring them in and out of the viewer’s conscious perception, with no need for cuts. “Parts of an image that are continuously present on the screen can be suppressed from the perceiver’s mind and then brought out at will, just like a magician’s rabbit pulled out of the hat,” Gepshtein anticipates. “In effect, one scene may emerge in the middle of the other without cuts, and without the artificial tools of image morphing or dissolves.”

This is hard to envision, and even Gepshtein has a hard time describing the future cinematic experience. But what if, instead of cutting to a new scene, the filmmaker swapped out one setting for another without us even noticing? Techniques reminiscent of this have already been used—albeit not with the subconscious seamlessness envisioned by Gepshtein—in movies like Speed Racer, from 2008.

Gepshtein believes the cinema of the future might even be a shared, immersive experience, one in which the events seem to unfold all around the viewer. “You could enter it like architecture, and there could be other people in the same space,” he says.

Because the science of vision is relevant to so many things, the potential applications of Gepshtein’s research range widely, from theater and virtual world-building in video games to highway signage, advertising displays, and building design. Architects are especially interested in researching the effect of their structures on human perception—something they’d like to know before they start building—so Gepshtein has received a small seed grant from the Academy of Neuroscience for Architecture, an organization that links neuroscience research to how we respond to the built environment.

Still, Gepshtein hopes especially to apply his maps of perception to film. For now, he recognizes that conveying his vision to working directors is going to be a challenge—“We don’t have a language for this,” he notes—but he believes a pilot visualization of the project would help overcome this barrier, and he is looking for funding to create a short demo reel.

Derrickson, the director, says he would love to see such a demonstration. “Somebody is going to make a movie at some point that does employ a new language,” he says. “I just can’t imagine what that would be, unless neuroscience brings in a new methodology of perception that somehow transcends our own way of looking at the world.”

That’s precisely what Sergei Gepshtein has in mind.


This post originally appeared in the May/June 2014 issue of Pacific Standard as “The Cut Stays Out of the Picture.” For more, subscribe to our print magazine.

Jennifer Ouellette
Jennifer Ouellette is a science writer based in Los Angeles. Her most recent book is Me, Myself and Why: Searching for the Science of Self.

A weekly roundup of the best of Pacific Standard and PSmag.com, delivered straight to your inbox.

Recent Posts

August 27 • 4:00 PM

The ‘Non-Lethal’ Flash-Bang Grenades Used in Ferguson Can Actually Be Quite Lethal

A journalist says he was singed by a flash-bang fired by St. Louis County police trying to disperse a crowd, raising questions about how to use these military-style devices safely and appropriately.


August 27 • 2:00 PM

Do Better Looking People Have Better Personalities Too?

An experiment on users of the dating site OKCupid found that members judge both looks and personality by looks alone.


August 27 • 12:00 PM

Love Can Make You Stronger

A new study links oxytocin, the hormone most commonly associated with social bonding, and the one that your body produces during an orgasm, with muscle regeneration.


August 27 • 11:05 AM

Education, Interrupted

When it comes to educational access, young Syrian refugees are becoming a “lost generation.”


August 27 • 9:47 AM

No, Smartphone-Loss Anxiety Disorder Isn’t Real

But people are anxious about losing their phones, even if they don’t do much to protect them.


August 27 • 8:00 AM

A Skeptic Meets a Psychic: When You Can See Into the Future, How Do You Handle Uncertainty?

For all the crystal balls and beaded doorways, some psychics provide a useful, non-paranormal service. The best ones—they give good advice.


August 27 • 6:00 AM

Speaking Eyebrow: Your Face Is Saying More Than You Think

Our involuntary gestures take on different “accents” depending on our cultural background.


August 27 • 4:00 AM

The Politics of Anti-NIMBYism and Addressing Housing Affordability

Respected expert economists like Paul Krugman and Edward Glaeser are confusing readers with their poor grasp of demography.


August 26 • 4:00 PM

Marching in Sync May Increase Aggression

Another danger of militarizing the police: Marching in lock step doesn’t just intimidate opponents. It impacts the mindset of the marchers.


August 26 • 3:03 PM

The Best Reporting on the Federal Push to Militarize Local Police With Riot Gear, Armored Vehicles, and Assault Rifles

A few facts you might have missed about the flow of military equipment and tactics to local law enforcement.


August 26 • 2:00 PM

How the Other 23 Percent Live

Almost one-fourth of all children in the United States are now living in poverty, an increase of three million kids since 2005.


August 26 • 12:00 PM

Why Sports Need Randomness

Noah Davis talks to David Sally, one of the authors of The Numbers Game: Why Everything You Know About Soccer Is Wrong, about how uncertainty affects and enhances the games we watch.


August 26 • 10:00 AM

Honor: The Cause of—and Solution to—All of Society’s Problems

Recent research on honor culture, associated with the American South and characterized by the need to retaliate against any perceived improper conduct, goes way beyond conventional situations involving disputes and aggression.



August 26 • 8:00 AM

The Transformative Effects of Bearing Witness

How witnessing inmate executions affects those who watch, and how having an audience present can also affect capital punishment process and policy.



August 26 • 7:15 AM

Being a Couch Potato: Not So Bad After All?

For those who feel guilty about watching TV, a new study provides redemption.


August 26 • 6:00 AM

Redesigning Birth Control in the Developing World

How single-use injectable contraceptives could change family planning in Africa.


August 26 • 4:15 AM

How Gay Men Feel About Aging

Coming to terms with growing old can be difficult in the gay community. But middle-aged men are inventing new strategies to cope.


August 25 • 4:00 PM

What to Look for In Dueling Autopsies of Michael Brown

The postmortem by Michael Baden is only the beginning as teams of specialists study the body of an 18-year-old African American killed by police in Ferguson, Missouri.


August 25 • 2:00 PM

Thoughts That Can’t Be Thought and Ideas That Can’t Be Formed: The Promise of Smart Drugs

Are we asking the right questions about smart drugs? Marek Kohn looks at what they can do for us—and what they can’t.


August 25 • 12:00 PM

Does Randomness Actually Exist?

Our human minds are incapable of truly answering that question.


August 25 • 10:31 AM

Cesareans Are Still Best for Feet-First Babies

A new study confirms that surgery is the safest way to deliver a breech fetus.


August 25 • 10:00 AM

What Can Hurricanes Teach Us About Socioeconomic Mobility?

Hurricane Katrina wrought havoc on New Orleans but, nine years later, is there a silver lining to be found?


August 25 • 8:00 AM

How Low Voter Turnout Helps Public Employees

To a surprising degree, as voter turnout goes down, public employee compensation goes up.


Follow us


Subscribe Now

Education, Interrupted

When it comes to educational access, young Syrian refugees are becoming a “lost generation.”

No, Smartphone-Loss Anxiety Disorder Isn’t Real

But people are anxious about losing their phones, even if they don’t do much to protect them.

Being a Couch Potato: Not So Bad After All?

For those who feel guilty about watching TV, a new study provides redemption.

How Gay Men Feel About Aging

Coming to terms with growing old can be difficult in the gay community. But middle-aged men are inventing new strategies to cope.

Cesareans Are Still Best for Feet-First Babies

A new study confirms that surgery is the safest way to deliver a breech fetus.

The Big One

One in two full-time American fast-food workers' families are enrolled in public assistance programs, at a cost of $7 billion per year. July/August 2014 fast-food-big-one

Copyright © 2014 by Pacific Standard and The Miller-McCune Center for Research, Media, and Public Policy. All Rights Reserved.