Rav Shlomo friefeld z”l is reported in his biography to have reacted two different ways to two different people. One talmid was a BT and began wearing chasidish clothing. He told him that he wasn’t ready, while another talmid who had grown up chasidish had shed his garb – the rosh yeshiva told him “you didn’t come here to fall”.
Dropping from one’s mesorah – whatever that may be – is far reaching. Rav tzadok hacohen writes in many places that Hashem cares more about where you’re heading than where you actually are. Rabbi yisroel reisman said on numerous occasions that when young couples decide to go against their upbringing in issues such as cholov yisroel, it’s building a house on spiritual deficiency and regression.
Avodas Hashem is all encompassing; it applies to all people on their level, but it is always the end all be all. No one “doesn’t fit in” to being an eved Hashem in all that they do.