My response to the first paragraph of your post is to strongly disagree with the premise that a home such as you describe is unlikely to produce a child who is a bad influence.
As far as the second part of your post, I would hope that not not only would such a person be ostracized, he would be, more importantly, watched very carefully.