I would like to share my own experience of this.
“I noticed many thoughts THAT ARISE in my mind are irrelevant talk”
We can hardly help that. The task consists of stopping those thoughts after they have arisen. And yet, at the same time, we CAN help that – for the more we train the mind to dwell on Dhamma, the more will the mind be fully occupied there, and those unwelcome thoughts can find no way in. The door is blocked. I cannot maintain this state of ‘guarding’ mindfulness all of the time, not to say most of the time.
For example, watching the world news – the bush fires in Australia. Taken by the ‘spectacular’ burning forests, I may want to watch the full report. Then..’what if I or a loved one were caught there? Could it happen here? What to do in such a situation?’ All irrelevant vaci sankhara. Suffice instead to spare a thought of genuine compassion. That should lead to Karuna and on to Metta in a general, universal sense. Then it is no longer vaci sankhara. On the contrary, what in the beginning could have led to harmful vaci sankhara has been turned into meritorious Metta.
Same when the arammana is an inner one, a mental one, as in the case you take up. In most cases, these are just random thoughts with no bases of ‘potential good’ inherent in them. Just mental babble. They are to be discarded, but we may not even become aware of that until some time has elapsed. But we get back on the right track alright.
And further to what Christian is saying – it just so happens (!!) that my latest contribution in Meaning of Key Pali words forum (December 31, 2019 at 3:58 pm) sn35.244 (States that entail Suffering) addresses specifically this question, right at the very last chapter.