I have a חזקה in this regard. I’m the third generation whose wife is taller than he is. Slightly older too, in my case.
My opinion? Why should it make a difference? No one blinks an eye if a 30 year old guy goes out with a 25 year old girl, why should the reverse be different?