Apushat- what in the world makes you think that other people view you as off the Derech?
You said yourself that you do Mitzvahs and follow your Rav.
Are you really narrowminded enough to think that if you don’t wear “the levush” that some people decided is right to wear, then you’re not as good of a Yid?