But if you’re asking the question that means there is something else that your thinking about but are having a difficult time (or just dont want to) verbalizing
that depends on how much you dont like the person’s looks. if you really can stand the way that person looks then it is only fair to say no because he/ she wont change. for instance if you really want a girl who is small and petite and this girl is tall and a bit broad then she is not for you. on the other hand if you want a girl who is light haired and this girl is dark then it is silly to say no based on that alone. shidduchim are more then looks, there is personality , hashkofos and interests that are more important and so what if you imagined you would marry a girl with blond or red hair, is that really so important? now when there is such a issue with shidduchim and i dont think its right to be sooo picky!!!
I always give it a second date unless I am turned off. Meaning that if I am just not attracted, I still try to say yes. I only say no about looks when I feel like there is an issue that sticks out as being unattractive.
I once heard a Rav (forgot who) quoted as saying that if one does not like their date’s looks, they should keep going until the 3rd date. If it still bothers you then, it’s time to break up. And don’t say “it’s just chitzoniyus, it doesn’t matter” – that’s what you should have been telling yourself for the past 10 years. But now, if there’s no attraction there’s no attraction.