Your “He got me scared, though…” was a bit ambiguous. It wasn’t entirely clear whether he got you scared that he said “I’m saying yes” to your face rather than the shadchan (and you thought that inappropriate) or if he got you scared that he was finished with you.
Anyways, why did you give him a long look? You really entertained the idea he might have been dumping you right to your face?