Ujm my source, as far as I can recall, was reliable but secondary, and I am not able to find the source right now. I was almost ready to agree with your claim, but see my other posts with quotes from Laws of Outreach sefer. Chazon Ish was strict on handshakes, but more lenient on Brochos. So, I’ll stand by my recollection of the story.
Also, see those quotes for multitudes of subtle psychological considerations and opinions on these issues – will the person be offended? is handshake cold/warm/hot potato(e)?
One more thought here: given variety of opinions, there is a greater danger here to use your bias to be either meikel or machmir and justify your own thinking by finding appropriate quotes. Just in an example above, one person might (incorrectly) learn that he’s following Chazon Ish by shaking hands, learning it from Kiddush, or similarly (incorrectly) conclude from not shaking hands to not making Kiddush.