We were not a nation until Matan Torah. Prior to that, we were tribes so if some tribes died, they were considered separate than us. Even after Mattan Torah,when we became a nation, people still held to the notion that you could only marry within your tribe. Cross marrying is unheard of! So as long as 4/5th were not part of my tribe, people did not care. In addition, not everyone worked. Shevet Levi was a complete tribe that did not work, but other tribes individual people had a good life. They went to the theaters of the goyim, and they moved out of Goshen and settled among the goyim.
Plegesh Bigeva, the whole tribe of Binyamin was almost wiped out but we do not have anything special. Even the Jews at that time did not care if Shevet Benyamin would be exterminated until it was brought to their attention that Klal Yisroel needs 12 tribes.
So by 4/5th dying, I do not see it a big deal among the survivors. It is not like today where we intermarry among various shevatim. Back them, you kept to your own.
A bigger question would be, the meforishim state that they died during this makka so not to make a chillul hashem. Imagine if the whole state of Israel, about 6 million people, just disappeared, wouldn’t that also be a chilul hashem? Who and where did they bury 6 million people in 3 days?