I’m not in line with the Brisker Rav or the Mishna Berurah.
The first Rambam is specifically about learning Torah. The second Rambam is about anyone who drops the mundane for serving Hashem.
The whole ‘in our times’ apologetics only speaks to one who is trying to pull things in line. As in how can one learn all day and be doing the right thing. Even according to the Rambam It seems to be possible. It still remains to wonder what was out of line for the Rambam. And was nothing out of line for the luminaries that we mentioned.