The main reason, is that it is not acting like an adult to not be honest and forthright with another person.
Be careful with that. Mutar l’shanos mipnei hashalom. You certainly wouldn’t expect someone to be honest and forthright if they didn’t want to continue dating because one found the other unattractive.
The most important thing to take into consideration is the other person’s feelings, which is actually a very good reason to use a shadchan.