Perception and Imagination: a Neuroscientist's Perspective

When you look at the image of the apple on your computer monitor, an apple exists in two places: as a picture on your monitor and as an activation pattern in your brain. If you close your eyes and imagine an apple, an apple exists, but only in one place — in your brain. That is the difference between perception and imagination.

It’s useful to have words to describe the various apples. The apple that is out in the is in the world we’ll call a real-world apple.The apple in the brain has two names that are slightly different. One is “mental model1”, which is the identification of an object in your mind; the second is “cell assembly” which is what many neuroscientists think of as the biological underpinnings of a concept or idea. A cell assembly is a set of neurons which, when activated as a group cause the corresponding “mental model” to spring to awareness.2 Bear in mind that the concept of cell assemblies is not proven, it is a hypothesis. Although no one has ever seen or demonstrated a cell assembly, evidence is accumulating.

Perhaps the clearest argument for the existence of cell assemblies comes from recordings from single neurons in animals and humans. Let’s imagine what we would see if we were able to record from all of the neurons in a someone's brain. When the person sees or imagines an apple, all of the neurons in the "apple" cell assembly will "fire" (have a high rate of action potentials) while other neurons are quiet. If a different object is seen or imagined, a different cell assembly would activate. This experiment is impossible with present technology, since, at most, about 100 neurons can be recorded at one time. We can, however, record up to about 100 neurons at a time in people or experimental animals. Here is what the cell-assembly model predicts during recording from a limited number of neurons: During presentation of most stimuli, all recorded neurons would remain quiet. On some occasions, a particular stimulus will evoke the firing of one of the neurons. The neuron that is activated will be activated whenever the subject sees the same object, but not other objects. The neuron will also fire when the subject thinks about the object.

This experiment was carried out by a group of scientists at UCLA and reported in 2009 in the journal Science.3 Dozens of neurons were simultaneously recorded in patients waiting for neurosurgery. In phase 1 of the experiment a subject was shown a wide range of video clips. A few of the neurons (about 10%) fired reliably to some of the video clips. In the first part of the experiment a patient was shown a series of video clips, to see if any of the neurons fired. In the second part of the experiment the patients was asked to recall from memory any of the video clips he had seen and say the name when he remembered them. The video below, from this paper’s supplemental material, shows results from a single neuron in both parts of the experiment. You hear a “pop” with each action potential. The scrolling white trace is a plot of the neuron's firing rate. This neuron is selectively activated by a video of “The Simpsons”. The first 3/4 of the video (up to 1:35) is from the first part of the experiment, when a series of video clips is shown. The final segment, from 1:35 until the end, is from the second portion of the experiment, when the subject was asked to recall the clips he had seen. You can see that this neuron is highly selective to “The Simpsons”, in both parts of the experiment.

Although this result is precisely what the cell assembly model predicts, it is not a proof that the cell assembly hypothesis is correct.


To my mind, the most impressive aspect of this result is the demonstration that the cell fired virtually identically during the perceptual and imaginative aspect of the experiment — remarkable evidence that identical cell assemblies/mental models are activated during sensory perception and imaginative recall. Supporting evidence comes from functional imaging studies where it is found that the brain areas activated during visual perception of an object largely overlap the brain areas activated when subjects are asked to imagine the appearance of the same object4.

While imagining the appearance of an apple may not seem very important, the general role of imagination in human thought is critically important, probably much more important for humans than other species. Imagination is the creation and manipulation of brain models when there is no "stimulus" in the current environment. We spend hours a day using imagination.

  • Planning a Route: Standing at a choice point, imagining various routes and selecting the best one.
  • Creating a Strategy: Having a task, and imagining various possible solutions until an optimal solution is found.
  • Recalling a Memory: For example, remembering the first day of school.
  • Reading and Enjoying a Novel: The written words of the novel elicit images and narrative of objects and events that are not present.
  • Being Creative: Any creative action involves imagining the outcome before creating the product. This applies to all aspects of endeavor, from art, to personal relations to science.
  • Understanding Historical Timelines: The timeline is in the mind, not in the world.
  • Day Dreaming
  • Hallucinating

The notions of Cell Assemblies and Mental Models are simplifications of very complex processes. Yet I feel they capture the essence of perception and imagination. They are the early steps in understanding how the brain works. While experiments support these conceptions, they are not explanatory. We do not understand the subjective experience of consciousness. Although the activation of a cell assembly leads to conscious perception, we have no idea how this happens.

Update, Jan 7, late:

Scott Barry Kaurman reminds me that different types of "mental models" are localized/processed in different brain regions. For example, the cell assembly for an object, such as an apple, would likely be in the temporal lobe, while the cell assembly for an imagined future possibility would likely be in the frontal lobe. In the language of functional imaging, these are described as different networks. by John Kubie

1 Reasonable equivalents for term “mental model”are the words “idea” and “concept”.

2 The term “Cell Assembly” was introduced by Donald Hebb in his famous 1949 book “The Organization of Behavior: A Neuropsychological Theory”.

3 The subjects were patient who suffered from epilepsy and were scheduled for surgery to treat their condition. Electrodes were implanted prior to surgery to guide surgery. Recordings were made prior to surgery. Internally Generated Reactivation of Single Neurons in Human Hippocampus During Free Recall Gelbard-Sagiv et al. Science 3 October 2008: Vol. 322 no. 5898 pp. 96–101

4 Brain areas underlying visual mental imagery and visual perception: an fMRI study. Ganis G, Thompson WL, Kosslyn SM. Brain Res Cogn Brain Res. 2004 Jul;20(2):226–41.

The Society for Neuroscience and its partners are not responsible for the opinions and information posted on this page. Terms & conditions.