Similarly, if the problem is that he got home after dinner and bedtime, the Rambam should lead the paragraph appropriately. And then add, that even a full time learner that does not discuss the latest news at the table or go to movies with his wife, loses his portion in the world to come because he was simply not spending enough time being worldly with the people closest to him.
So it must be something else. What does the Rambam say this person is doing so wrong?