Vision Is a Series of “Snapshots” – Movie Analogy

October 13, 2019

Vision – How Do We See an Object?

1. Vision or “seeing” appears to us as continuous. We see people moving around, vehicles moving, animals running around, etc. However, in reality, “seeing” happens due to a series of “snapshots” that our physical eyes take. Please bear with me as I set the stage with the following Pāli terms. It is not necessary to know these Pāli terms in detail, but try to get the basic idea.

  • A key idea behind Buddha Dhamma is that we experience only one citta (loosely translated as a thought) at a time and that citta is focused on ONE ārammana. In other words, while the mind is registering a visual event, it cannot hear, smell, taste, or feel a touch. The keyword ārammana was introduced in the post, “Vipāka Vēdanā and “Samphassa jā Vēdanā” in a Sensory Event.”
  • Even when we focus on “seeing,” it does not happen continuously. The mind can process only one cakkhudvāra citta vithi (with 17 cittā) at a time. The mind processes that cakkhudvāra citta vithi with three more manōdvāra citta vithi. At the end of those citta vithi, the mind has captured a ‘snapshot” of the object and recognized it. That is one “snapshot” of a moving object.
  • Our “seeing of a moving external object” within a few seconds involves many such “snapshots.” Our perception of a moving object is the result of all those “snapshots.” We do not see the individual “snapshots.”
Movie Analogy – Series of Snapshots

2. We can simplify and understand the above process using an analogy. What I stated above is — in principle — what happens when we watch a movie.

  • When making a movie, a video camera captures many static pictures (snapshots) of a scene. Then those snapshots are projected to a screen at a certain rate. If the playback speed is too slow, we can see individual pictures, but above a certain “projection rate”, it looks like real motion. Here is a video that illustrates this well:

  • A movie projector projects static pictures to the screen at a rate of about 30 frames a second, and we see the movie as a continuous progression of events. If the projection rate is low, we can see it frame by frame or as individual “snapshots”. When projected at 30 frames a second, we do not perceive those static pictures. Then we perceive a continuous progression without any gaps.
  • More details in the post, “Citta and Cetasika – How Vinnana (Consciousness) Arises.”
  • That is why the Buddha said that the mind (or viññāṇa) is a magician. We perceive a streamlined world, even though the reality is that our sensory faculties detect only a series of “snapshots,” It is the mind that conceals the reality and gives us a perception of a continuous progression of events.
  • It is critical to understand this point. It helps getting rid of sakkāya ditthi, as we will discuss in the next post.
Mind and the Brain – Two Different Entities

3. In an early post on this series, I pointed out that cakkhāyatana is cakkhu pasāda rūpa, not the physical eyes. See, #12 of “Buddhist Worldview – Introduction.”

  • That cakkhu pasāda rūpa (or simply cakkhu) is part of the gandhabba, our “mental body.” The gandhabba has the seat of the mind (hadaya vatthu) surrounded by the five pasāda rūpa corresponding to vision, hearing, taste, smell, and touch.
  • When our physical eyes capture an image of an external object, that image goes to the visual cortex in the brain. The signal is processed there and then transmitted to the cakkhu pasāda rūpa, which then makes contact with the hadaya vatthu. That contact (phassa) leads to the arising of cakkhu viññāna at the hadaya vatthu. More details at, “Brain – Interface between Mind and Body.”
  • By the way, that is the step, “cakkhuñca paṭicca rūpe ca uppajjati cakkhuviññāṇaṃ” discussed in #7 in the post, “Contact Between Āyatana Leads to Vipāka Viññāna.”

4. Therefore, the brain is like a computer that helps convert the image sent by the physical eyes to a form that can be sensed by the hadaya vatthu, the seat of the mind. Therefore, vision involves a somewhat complex process.

  • Similar processes take place for the other four physical sensory events. For example, when the physical ears capture a sound, that signal goes to the auditory cortex in the brain for processing. That signal then goes to the sōta pasāda rūpa, which in turn makes contact with hadaya vatthu to transfer. That gives rise to sōta viññāna via, “sōtañca paṭicca sadde ca uppajjati sotaviññāṇaṃ.”
Reviewing the Whole Series Could Be Helpful

5. It may need some effort to understand this sequence of events. But that is really necessary in order to comprehend the overall process before we get to the next post.

  • It is a good idea to print all the posts in the subsection, “Worldview of the Buddha” up to now and go over them carefully. There are about eight posts up to now.
  • It is not necessary to understand the DETAILS of #6 and #7 below. But it is good to get the general ideas involved. I am providing this information to show that new findings in science are not only compatible with Buddha Dhamma but also help explain the key concepts in Buddha Dhamma.
The Brain Processes Visual Signals at About 30 Frames per Second

6. A recent study has reported that the minimum time needed for recognition of a static picture is about 13 milliseconds (Ref. 1). That means we should be able to see such snapshots projected at 77 frames per second at the highest rate. However, that is probably “pushing it” and not comfortable for the brain to handle. That is probably why movies use a projection rate of about 30 frames per second as mentioned in #2 above.

  • It is interesting to note that the time for neural information to reach the brain takes about 15 to 30 milliseconds (References 49, 50 in Ref. 2). Therefore, a projection rate of 30 to 50 frames is compatible with that measurement too.
  • A millisecond is a thousandth of a second.
Same Analysis Holds For Other Four Physical Senses

7. A similar set of rules are valid for hearing as well. Another recent study (Ref. 2) found that sounds could be recognized at rates up to 30 sounds per second. That corresponds to a “sound packet” of a duration of about 33 milliseconds that can be detected and recognized.

  • However, people speak at a much slower rate of 150 words per minute. That is about 2 words per second, much less than 30 possible words per second that would be possible according to the above study. So, there is no problem with hearing what other people speak, even if someone talks faster than the average rate.
  • There are no studies available at this time from science for the other three sensory events (taste, smell, and body touches). But the same process holds for those as well.
Aside – Cognition (Saññā) Requires More Than Detection

The following points (#8, #9) are “asides”. It is information that is not necessary but could help those with familiarity with Abhidhamma.

8. We must keep in mind that “experiencing a sensory input” is much more complex than just receiving that sensory input. For example, the mind needs to not only see an object or hear a sound, but it needs to recognize what it is and also need to generate a vēdanā.

  • As an example, when the sound “apple” is heard, the mind needs to know what an “apple” is. Someone who does not speak English would not know what is meant by the word “apple”. But for those who speak English AND have had experience of eating apples would have MEMORIES of those. Therefore, the mind needs to compare the received sensory with past memories to recognize what it is!
  • The mind does that very fast with the help of the manasikāra cētasika. As you may know, manasikāra is one of the seven universal cētasika that arises with each citta. Thus, the mind is able to recognize a sensory input instantaneously, as soon as it receives a “data packet.”
  • More details at, “Citta and Cetasika – How Vinnana (Consciousness) Arises.”
Aside – Process In Abhidhamma Language

9. Actual “seeing” or vision takes place at hadaya vatthu. Same for the other four types of sensory events. For example, let us consider a “packet of data” sent from the physical eye to the brain. The brain processes that information and transmits to the cakkhu pasāda. As you may remember, the five pasāda rupā (cakkhu, sōta, ghāna, jivhā, kāya) surround the hadaya vatthu. Now the cakkhu pasāda makes contact with the hadaya vatthu by hitting it. That causes the hadaya vatthu to vibrate 17 times, much like a gong hit by an iron rod vibrating for a certain fixed number of times.

  • The 17 vibrations of the hadaya vatthu correspond to the 17 cittā in a citta vithi. Such a citta vithi is a pancadvāra citta vithi because one of the five physical senses or pancadvāra (“panca” or five + “dvāra” or “door”) initiates it.
  • Imagine a blade clamped at one edge and hit on the un-clamped side. The blade will vibrate. It vibrates for a certain FIXED number of times. For a given material, that number is fixed. The same thing happens when a pasāda rūpa makes contact with the hadaya vatthu. The hadaya vatthu vibrates 17 times, with each vibration leading to the arising of a citta. That is the origin of a citta vithi with 17 cittā. Those 17 vibrations are a form of energy called a hadaya rūpa.

10. The misconception that any rūpa has a lifetime of 17 thought moments arose because of not understanding the difference between a rūpa (which is the image of an external object) and a hadaya rūpa (which is just the 17 vibrations of the hadaya vatthu).

  • In other words, this information packet is received and processed by the hadaya vatthu within those 17 cittā. The information is complete by the fourth citta (fourth vibration of the hadaya vatthu) and then the rest of the citta in that citta vithi deal with this information. Three more citta vithi run by the hadaya vatthu itself completes the process. Those additional citta vithi, initiated by the mind, are manōdvara citta vithi. Here, manōdvara means the “mind-door.”
  • Details of #9 and #10 at, “Does any Object (Rupa) Last only 17 Thought Moments?.”
The mind is Fast But the Brain is Slow

11. Thus we can see that there is a vast difference in time between the two processes involved. The physical body acquiring data takes the time of the order of 10 milliseconds. The mind processing that information within a billionth of a second (using one pancadvāra citta vithi and three manōdvara citta vithi.)

  • Even if the five senses keep sending data continuously, the mind is “just sitting there” most of the time. Let us examine this in a bit detail: Suppose the brain keeps sending data from the eye non-stop. Since each “packet” takes, say ten milliseconds, then in a second, there will be 100 “data packets” of vision coming in. If the brain is going at full speed, it can send at most 500 (=100×5) “data packets” from all five physical senses in a second. Then the mind will be spending less than a millionth of a second in processing all that data. During a movie that lasts two hours, the mind will be active probably less than a second in total. Of course, this is a very simple analysis and the actual values could be higher, especially when taking into account the interaction of dhammā with the mind directly, i.e., “manañca paṭicca dhamme ca uppajjati manoviññāṇaṃ.”
  • During those gaps, the hadaya vatthu also interacts (both ways) with the mana indriya in the brain. In particular, it gives instructions to the brain (via mana indriya)on how to control the physical body in response to the sensory inputs.
  • Thus, the mind (or more precisely the hadaya vatthu) is just sitting there most of the time. That “idle state” of the mind is the “bhavānga” state.
  • A key point here is that the mind spends only a VERY SHORT TIME experiencing the SENSORY INPUTS. There is no “self” watching a movie. However, the mind gives the illusion that there is a “self” watching the movie.
  • The above is a very brief discussion. Of course, there are more details, but one can hopefully get the basic idea. Please ask questions if something is not clear. It is critical to understand this post.
Summary

12. The critical point embedded in the Chachakka Sutta (MN 148) is that there is no “self” experiencing the external world. We have been discussing the initial steps in sensory events addressed by that sutta.

  • The key message in the sutta is that the mind DOES NOT experience the external world CONTINUOUSLY. Instead, the mind is active only for very brief periods of time when receiving inputs from the five pasāda rūpa. As we above it is the brain that is “on” much longer than the mind. Once the brain processes information packets, the mind absorbs that information within a “blink of an eye.”
  • On the other hand, the brain has a heavy workload while watching a movie. It has to process audio and video inputs at a rapid rate for the duration of the movie. One could get a headache if one watches two movies at a stretch. But even during that time, the mind is mostly in the bhavānga state. There is no “self” watching the movie. It is just a series of events taking place. The mind is “putting all those “events” together and giving the appearance of a continuous progression of events. Thus one has the perception that “I am watching a movie.”
  • We will discuss that in detail in the next post. Later on, we will also discuss why it is also incorrect to say that there is “no-self.”

REFERENCES

  1. M. C. Potter et al., “Detecting Meaning in RSVP at 13 ms per Picture”, Journal of Cognitive Neuroscience, vol. 13, pp. 90-101 (2014).
  2. V. Isnard et al., “The time course of auditory recognition measured with rapid sequences of short natural sounds“, Scientific Reports, vol. 9, pp. 1-10 (2019).

Click on the links to download the publications.

Print Friendly, PDF & Email