If you wear jeans just say you wear them because you like them, don’t give an explanation. They are not appropriate attire for religious people as they are a garb of non Jews but hey if you want to wear them go ahead just don’t go on the defense.
I would like to know where you get that its not appropriate for religious people. Is it that its casual? Because then women would not be able to wear jean skirts and I never heard of them being outlawed (frowned upon in some communities, but not outlawed). SOMEONE PLEASE EXPLAIN WHY WOMEN CAN WEAR JEAN SKIRTS BUT MEN CANNOT WEAR JEANS (pardon my yelling, but I broached this before in this topic and no one answered).
Also, everyone talks about dressing for our King – yes we do. But there is also a level up to dress. We generally save our fancier attire for shabbos/yom tov and then even fancier attire for weddings. If you wear it all week, then on shabbos its not a special garment! So yes, I see nothing wrong with being less formal during the week.