The way I and any other lubavitcher would understand your post is as follows:
You should reconsider accepting what a tzadik gamur says because a bunch of reshaim vtov lahem (again Tanya definitions here, I would be in the same category) have difficulty with what he says.
Hmm the polite response would be “bmakom gedolim al taamod”