I guess that is possible.
However, kama raga and patigha go together, just like dosa (dvesha) is the second manifestation of lobha (in simple terms, one gets angry because he/she is not getting what is desired).
So, it is more likely that kama raga and patigha are removed at the same time.
