“or to cloister yourself lifnai v’lifnim in the kodesh hakedoshim”
Only one person was allowed into the kodesh hakedoshim, and only on one day.
In other areas of Kodesh only Kohanim were allowed.
The Tribe of Levi at Exodus was only about 1/40th total Jewish nation.
Most of us have to operate in the world of Chol and elevate it.
Today we are not lacking the “Tribe of Levi” who are most energized in the service of G-d. We are lacking the other tribes.