Yes, y not. You need to see the difference between “rupa” and rupakkhandha. Same for other four khandhas (with the exception of part of vinnanakkhandha; see below.)
See, “Difference Between Rupa and Rupakkhandha.”
The vinnanakkhandha is a bit different. All other khandha just go to “memory records” or “nama gotta” as they go to the past.
– Kamma vinnana are (only associated with mano vinnana) have energies. They are basically kamma bija.
– The other five types of vinnana (cakkhu, sota, ghana, jivha, kaya) come to the mind momentarily and then recorded as past events. Same for “vipaka mano vinnana”.
– See, “Viññāna – What It Really Means.”
Also, even in kamma vinnana. it is not just one. There can be numerous kamma vinnana (kamma bija) waiting to bring kamma vipaka (including many possible future existences.) Those kamma vinnana that can bring future existences are also called “patisandhi vinnana.”
– One needs to spend time, contemplating and grasping the ideas.
Also note that we cannot recall all our memories, especially from past lives. Some children can just recall the immediate previous life.
– But by cultivating abhinna powers one can recall many past lives. The Buddha could recall as many as he wished.