DY: I’ve read a lot of the frum vegetarian literature, and I can assure you they are mainly coming from a Torah perspective (concerns about the rampant and unnecessary cruelty in the living conditions of animals) rather than from a anti-Torah perspective (that we have no right to eat animals or that shechita itself is cruel or wrong). Even the most extreme (who think the whole world should stop eating meat) are making arguments that are completely consistent with those of Rav Kook, who forsaw a global return to vegetarianism (as part of the Messianic age), and thought that only the flour korbanos would be restored in the third Beis haMikdash.
It’s possible that some are really coming from an anti-Torah perspective, but here’s a way to find out: ask them if they would eat from the korbanos if moshiach came and ruled that they were required to (but wasn’t going to punish them if they did not). I bet most, perhaps nearly all, frum vegetarians would eat it.
Popa, vegetarians have a different view of the halacha than you. Many authorities hold that it is forbidden to cause additional suffering to animals for financial gain (see my sources above), and the main reason for the conditions on factory farms today is the desire to reduce costs and increase profits. So there is a good halachic argument that the meat industry today is based on violations of halacha and should not be supported.
And anyway, as noted above, it is totally normal to go beyond the letter of the law to avoid even the possibility of a transgression, or to further the purpose of the underlying law (to reducing animal suffering), as shown by the example of the charedi Rav Yitzchak Weiss cited above. From a Torah perspective that’s a lot more praiseworthy than the opposite: bending over backwards to avoid even the slight possibility that someone might think you’re a “tree hugger” or guilty of sympathizing with an “ism” other than Republicanism or conformism.