There is nothing particularly Jewish or Torahdik about Chassidish garb…
Yes, assimilation is not good. But just look around you– Jews don’t assimilate just because they shop at Target or TJ Maxx. You can choose to wear polo shirts and khakis, or knit tops and jean skirts, and not lose anything of your Yiddishkeit. It doesn’t make you “sympathize” with the secular world or lose your connection to Hashem.