It is true that those who CAN remember past lives are basically children under the age of 12 years or so.
– As a child grows, memories of the past life fade away.
In many of those accounts, they died when they were not necessarily children, but at older ages too; “Evidence for Rebirth“.
P.S. However, it is true that it is rare that the person died at old age in the previous life (most cases are of dying young or at middle age). This is also something to do with the memory getting weak at old age. So, the gandhabba that comes out may not have a good memory.