Styles constantly change. Right now black fedoras are “in”. Maybe in 100 years berets will be in (they are much more practical when davening), or Homburgs (better in the rain).
Fashion isn’t a matter of halacha. It’s a matter of anthopology.
If hats become “in” among the goyim, again, we’ll feel less self-conscious about wearing hats.