A couple of issues here:
1) there are people who do shake hands. If you say your religion does not allow it, then the person may think either your are a faker, or the one who does shanke hands is a faker.
2) whatever you say, it could reasonably be interperted that you or your religion has something against females, and if the person is not jewish, and even worse, if she is black, could be interpreted as being against people in one or both of the above 2 categories.
3) “my wife doesnt allow it” sounds that you are married to a nut-case
I feel the best approach :
1) realize that it is a very uncomfortable situation – for chizuk carry in your mind a “dimus de’yokno” of your rosh yeshiva, or maybe your chavrusah, or maybe even the best – your wife. Would you shake hands with the lady if they were present? (derech agav, the is a great shemirah against being nichshal in pe-ritzsus in the workplace – would you joke around with a lady colleague if your wife was present?
2)be mispallel that all should go well
3) try to avoid – carry stuff in your hands
4)if a hand is strecthed out, then say: “Among orthodox Jews there are various traditions, and by some, even men and women in the same family do not shake hands”. I beleive this statement quickly and effectively addresses the issues.
Hatzlacha ve’chol tuv