I’m not putting words in your mouth, and not attacking you. I’m simply telling you the logical conclusion to your argument to show you that you’re wrong.
What you just said now doesn’t make sense or is incomplete.
Whether the Rema would have come to his sevara without the minhag being in place is immaterial. The question you still haven’t answered (because the answer shows you’re wrong) is WHY those holy Jews stopped duchaning – specifically, was it a legitimate halachic reason, or not?