Rebbi Avihu asks in the midrash, doesn’t the expression, all men should not be in ohel moed, also include the Kohen Gadol himself? The Yerushalmi adds in his name that includes a malach looking like a human being. So, he answers that his face became kelapid aish, like a flame of fire. Maybe, this explains why by the second beis hamikdash more than three hundred kohanim gedolim where found dead when they were pulled out. They were not worthy to this conversion like a saraf. so they violated the above commandment and ended up dead.