@ Joseph
Eating Cholov Akum definitely causes negative spiritual consequences. And many hold that which is called Cholov Stam is Cholov Akum mamish.
Going against the will of Hashem causes negative consequences. Not just spiritual, ultimately in the olam hagashmi as well. Relying on the psak of the Posek HaDor can hardly be construed as going against th
You do know that there is Halachic basis for assuring most dairy products in America today (including the chalav Yisroel ones. This is why we have Poskim.