My rov told us in no uncertain terms that to speak while someone is sanctifying Hashem’s name b’rabbim by saying kaddish, brings mamesh a klala on him. I once told this over on this forum, that there is a rock somewhere in E”Y, I cannot remember where he told me it was, that my Rov said states that one who speaks during Kaddish brings a curse upon himself and loses his chelek in Olam Haba. I don’t know if it is true or not that this is the onesh, but it speaks to the severity of talking during kaddish.
