Charedi birthrate is probably higher than either DL or MO (though some MO and DL certainly have a lot of children), but this isn’t a mark of legitimacy.
Even if you argue that financial considerations are not a factor in whether to have another child, it’s hard to argue that mental health and psychological capacity should not be a consideration.
Some charedim may be having more children than they can psychologically handle (ie, having more children even though you’re already having nervous breakdowns or feel like you’re about to), just because it’s expected as a sociological matter in one’s community that each family has a large number of children.
Also note that before the 20th century, it’s very hard to find examples of gedolim having numerous children (even counting those who didn’t survive).