My point is not that you should stop because it’s wrong.(although I believe it is, as I mentioned.) My point is: why don’t you get a feel for what it means to stop a pleasurable behavior cold turkey, before calling others who have not risen to the challenge OTD?
Either way, having a hard time with one mitzvah does not make someone off the derech.