Ironically, I was at Chuck E. on Pesach. However, maybe it is true that it’s not a place to be on Chol Hamoed Pesach. Maybe if you are in a place where a hat and jacket look so incongruous, it’s not the attire that is a problem but the place you are at. An extreme example would be a movie theater. Sometimes there are a lot of non-Jews in CEC and maybe a hat and jacket type shouldn’t be there mixing with the goyim.
oomis- how many is an awful lot of men dressed in the so-called proper manner…? Although the numbers of such people are unfortunately growing it is still negligible. What about men who dress like your son? Are there none of them in unsavory places? There are bad apples in every lot.