I think having another woman, best if its the rebbitzen of the shul, go over and tell her straight out that she may not be aware of the halachic guidelines in the shul and please dress appropriately. It’s been done in the past. I would imagine its the same idea as a BY. When a student even wears the wrong sweater, there is a knas or some sort of warning before the knas.
Sometimes the best way is the blunt way.
WADR, being confrontational will only push the women in the opposite direction. If your goal is to make those who are not at your standards no longer Frum and thereby remove them from your shul, this would be a good idea.