How, Exactly, Do Our Brains Construct Reality?

In order to tell the story of your life, your brain needs to conjure up a world for you to live inside, with all its colors and movements and objects and sounds. Just as characters in fiction exist in a reality that’s been actively created, so do we. But that’s not how it feels to be a living, conscious human. It feels as if we’re looking out of our skulls, observing reality directly and without impediment. But this is not the case. The world we experience as “out there” is actually a reconstruction of reality that is built inside our heads. It’s an act of creation by the storytelling brain.

This is how it works. You walk into a room. Your brain predicts what the scene should look and sound and feel like, then it generates a hallucination based on these predictions. It’s this hallucination that you experience as the world around you. It’s this hallucination you exist at the centre of, every minute of every day. You’ll never experience actual reality because you have no direct access to it. “Consider that whole beautiful world around you, with all its colors and sounds and smells and textures,” writes the neuroscientist and fiction writer Professor David Eagleman. “Your brain is not directly experiencing any of that. Instead, your brain is locked in a vault of silence and darkness inside your skull.”

This hallucinated reconstruction of reality is sometimes referred to as the brain’s “model” of the world. Of course, this model of what’s actually out there needs to be somewhat accurate, otherwise we’d be walking into walls and ramming forks into our necks. For accuracy, we have our senses. Our senses seem incredibly powerful: our eyes are crystalline windows through which we observe the world in all its color and detail; our ears are open tubes into which the noises of life freely tumble. But this is not the case. They actually deliver only limited and partial information to the brain.

Take the eye, our dominant sense organ. If you hold out your arm and look at your thumbnail, that’s all you can see in high definition and full color at once. Color ends 20 to 30 degrees outside that core and the rest of your sight is fuzzy. You have two lemon-sized blind spots and blink fifteen to twenty times a minute, which blinds you for fully 10 per cent of your waking life. You don’t even see in three dimensions.

How is it, then, that we experience vision as being so perfect? Part of the answer lies in the brain’s obsession with change. That large fuzzy area of your vision is sensitive to changes in pattern and texture as well as movement. As soon as it detects unexpected change, your eye sends its tiny high-definition core—which is a 1.5-millimetre depression in the centre of your retina—to inspect it. This movement—known as a “saccade”—is the fastest in the human body. We make four to five saccades every second, over 250,000 in a single day. Modern filmmakers mimic saccadic behavior when editing. Psychologists examining the so-called ‘Hollywood style’ find the camera makes ‘match action cuts’ to new salient details just as a saccade might, and is drawn to similar events, such as bodily movement.

The job of all the senses is to pick up clues from the outside world in various forms: lightwaves, changes in air pressure, chemical signals. That information is translated into millions of tiny electrical pulses. Your brain reads these electrical pulses, in effect, like a computer reads code. It uses that code to actively construct your reality, fooling you into believing this controlled hallucination is real. It then uses its senses as fact-checkers, rapidly tweaking what it’s showing you whenever it detects something unexpected.

It’s because of this process that we sometimes “see” things that aren’t actually there. Say it’s dusk and you think you’ve seen a strange, stooping man with a top hat and a cane loitering by a gate, but you soon realize it’s just a tree stump and a bramble. You say to your companion, ‘I thought I saw a weird guy over there.’ You did see that weird guy over there. Your brain thought he was there so it put him there. Then when you approached and new, more accurate, information was detected, it rapidly redrew the scene, and your hallucination was updated.

If our senses are so limited, how do we know what’s actually happening outside the dark vault of our skulls?

Similarly, we often don’t see things that are actually there. A series of iconic experiments had participants watch a video of people throwing a ball around. They had to count the number of times the ball was passed. Half didn’t spot a man in a gorilla suit walk directly into the middle of the screen, bang his chest three times, and leave after fully nine seconds. Other tests have confirmed we can also be ‘blind’ to auditory information (the sound of someone saying “I am a gorilla” for nineteen seconds) as well as touch and smell information. There’s a surprising limit to how much our brains can actually process. Pass that limit and the object is simply edited out. It’s not included in our hallucinated reality. It literally becomes invisible to us. These findings have dire potential consequences. In a test of a simulated vehicle stop, 58 per cent of police trainees and 33 per cent of experienced officers “failed to notice a gun positioned in full view on the passenger dashboard.”

Things naturally become worse when our fact-checking senses become damaged. When people’s eyesight develops sudden flaws, their hallucinatory model of reality can begin to flicker and fail. They sometimes see clowns, circus animals and cartoon characters in the areas that have gone dark. Religious people have apparent visitations. These individuals are not “mad” and neither are they rare. The condition affects millions. Dr Todd Feinberg writes of a patient, Lizzy, who suffered strokes in her occipital lobes. As can happen in such cases, her brain didn’t immediately process the fact she’d gone ‘suddenly and totally’ blind, so it continued projecting its hallucinated model of the world. Visiting her hospital bed, Feinberg enquired if she was having trouble with her vision in any way. “No,” she said. When he asked her to take a look around and tell him what she saw, she moved her head accordingly.

“It’s good to see friends and family, you know,” she said. “It makes me feel like I’m in good hands.”

But there was nobody else there.

“Tell me their names,” said Feinberg.

“I don’t know everybody. They’re my brother’s friends.”

“Look at me. What am I wearing?”

“A casual outfit. You know, a jacket and pants. Mostly navy blue and maroon.”

Feinberg was in his hospital whites. Lizzy continued their chat smiling and acting “as if she had not a care in the world.”

These relatively recent findings by neuroscientists demand a spooky question. If our senses are so limited, how do we know what’s actually happening outside the dark vault of our skulls? Disturbingly, we don’t know for sure. Like an old television that can only pick up black and white, our biological technology simply can’t process most of what’s actually going on in the great oceans of electro-magnetic radiation that surround us. Human eyes are able to read less than one ten-trillionth of the light spectrum. “Evolution shaped us with perceptions that allow us to survive,” the cognitive scientist Professor Donald Hoffman has said. “But part of that involves hiding from us the stuff we don’t need to know. And that’s pretty much all of reality, whatever reality might be.”

The only thing we’ll ever really know are those electrical pulses that are sent up by our senses.

We do know that actual reality is radically different than the model of it that we experience in our heads. For instance, there’s no sound out there. If a tree falls in a forest and there’s no one around to hear it, it creates changes in air pressure and vibrations in the ground. The crash is an effect that happens in the brain. When you stub your toe and feel pain throbbing out of it, that, too, is an illusion. That pain is not in your toe, but in your brain.

There’s no color out there either. Atoms are colorless. All the colors we do “see” are a blend of three cones that sit in the eye: red, green and blue. This makes us Homo sapiens relatively impoverished members of the animal kingdom: some birds have six cones; mantis shrimp have sixteen; bees’ eyes are able to see the electromagnetic structure of the sky. The colorful worlds they experience beggar human imagination. Even the colors we do ‘see’ are mediated by culture. Russians are raised to see two types of blue and, as a result, see eight-striped rainbows. Color is a lie. It’s set-dressing, worked up by the brain. One theory has it that we began painting colors onto objects millions of years ago in order to identify ripe fruit. Color helps us interact with the external world and thereby better control it.

The only thing we’ll ever really know are those electrical pulses that are sent up by our senses. Our storytelling brain uses those pulses to create the colorful set in which to play out our lives. It populates that set with a cast of actors with goals and personalities, and finds plots for us to follow. Even sleep is no barrier to the brain’s story-making processes. Dreams feel real because they’re made of the same hallucinated neural models we live inside when awake. The sights are the same, the smells are the same, objects feel the same to the touch. Craziness happens partly because the fact-checking senses are offline, and partly because the brain has to make sense of chaotic bursts of neural activity that are the result of our state of temporary paralysis. It explains this confusion as it explains everything: by roughing together a model of the world and magicking it into a cause-and-effect story.

One common dream has us falling off a building or tumbling down steps, a brain story that’s typically triggered to explain a “myoclonic jerk,” a sudden, jarring contraction of the muscles. Indeed, just like the stories we tell each other for fun, dream narratives often centre on dramatic, unexpected change. Researchers find the majority of dreams feature at least one event of threatening and unexpected change, with most of us experiencing up to five such events every night. Wherever studies have been done, from East to West, from city to tribe, dream plots reflect this. “The most common is being chased or attacked,” writes story psychologist Professor Jonathan Gottschall. “Other universal themes include falling from a great height, drowning, being lost or trapped, being naked in public, getting injured, getting sick or dying, and being caught in a natural or manmade disaster.”

So now we’ve discovered how reading works. Brains take information from the outside world—in whatever form they can—and turn it into models. When our eyes scan over letters in a book, the information they contain is converted into electrical pulses. The brain reads these electrical pulses and builds a model of whatever information those letters provided. So if the words on the page describe a barn door hanging on one hinge, the reader’s brain will model a barn door hanging on one hinge. They’ll “see” it in their heads. Likewise, if the words describe a ten-foot wizard with his knees on back to front, the brain will model a ten-foot wizard with his knees on back to front. Our brain rebuilds the model world that was originally imagined by the author of the story. This is the reality of Leo Tolstoy’s brilliant assertion that “a real work of art destroys, in the consciousness of the receiver, the separation between himself and the artist.”

__________________________________