Those people are people who want to be with Jews or just check out davening for Shabbos. The rov just might want to invite those people over in the future if he can and if they return so that they don’t have to drive. He’s also avoiding the aveiro of embarrassing someone in case they do know they’re not supposed to drive.
In many cases, the driver barely knows he’s doing an aveirah by driving or that davening is a mitzvah. The rov is trying to prevent future aveiros and there is no mitzvah to tell someone to leave a shul because he drove. (Actually, in prior years our rov has told people not to drive on YK, to the point that shul attendance declines every year.)
Sam, see HaLeiVi’s post. He explains things better than I can because I am noigea bedovor and “in the parsha.”