This issue has always struck me as a Brisker kashea. Is the mitzvah of tzedakah a din in the cheftza or in the gavra? If the din is for latter, than one shoud give what one can and if the collector is a fraud, that’s his avlah. On the other hand, a din in the cheftza means that tzedaka money is mamon shel Heqdash and the donor acts as an apitrapos of Heqdash and must donate only to those peope or institutions that he (or she) is certain is worthy.
In practical application (not usually considered in Brisk), I think it’s a ittle of both. One should almost always give something, at least a minimum amount, and certainly never send anyone away from your door empty handed, but one shoud reserve one’s significant donations to people and institutions he knows are worthy.