Hachnosas Orchim, according to the Sefer Hachaim (brother of the Maharal) and the Maharil is not fulfilled by having a guest who comes to your door seeking a meal. the Maharil adds that if the guest is poor, one does not fulfill the mitzvah at all. It is tzedokoh and chesed, both precious mitzvos. But Hachnosas Orchim is done by proactively inviting the guest – as Avrohom Avinu did sitting outside seeking a guest to invite. And inviting even the gvir, the wealthy person is the mission of Hachnosas Orchim. It should not be confused with tzedokoh and chesed, also beautiful and precious mitzvos.
Check these references, and then weigh in again about the notion that the mitzvah is for the lonely and downtrodden.