Yash don’t put too much pressure on you. At the time of Lord Buddha, there were two brothers Purana and Isadatta who were sakadagamin. Purana was single and abstained from all sexual activity. Isadatta was still married and engaged in sex.
Lord Buddha said that both of them were reborn in a deva realm when they died. Sexual attraction depends on each person’s gati at these stages (puthujunas, sotāpanna and sakadagamin). Sooner or later you will lose this desire whether you like it or not. It is inevitable (anatta) that you become anagami.
From the moment you became sotāpanna the process began. The samyojana kāma ragā and dosa will break on their own. Of course if you want to progress faster, the practice of jhanas and the anagarika lifestyle are excellent ways.
See Migasālāsutta