This definitely sounds like you “are NOT attracted” to this guy and not that “you don’t feel an attraction” for this guy. There is a huge difference.
There are things about this guy you don’t like and that is reason enough to say “NO”. There is no point in wasting your time or his when you already find things that you don’t like. You will not change your mind, you will only look for more things not to like about him. I only say “if there is nothing wrong keep going”. Obviously this doesn’t apply here.