I know plenty of men who went to co-ed colleges or universities and remained strong in their Frumkeit. I also know some who went off the derech. Fortunately, those are few.
The difference, at least in the people I know, is the family. If they come from a strong family, with solid Torah values, they don’t have a problem. But if the family is dysfunctional, or if their practice of Judaism is inconsistent in areas, that’s where the problem comes in. But those people would probably go off the derech without college.