Sam2:
The response that I have heard people give to your question is that yes theoretically the Gedolim could be wrong/make a mistake, but in practice WE are not allowed to decide that they are wrong. I’m not saying I agree, but this is something I’ve heard.