Yitay,
With respect to hair covering and arms it is in the Gemara in Kesubos 72a-b. It is also clear from that mishna and Gemara that anything that Bnos Yisroel have a minhag to keep covered must be kept covered.
There is an issur brought down in the Gemara (Shabbos 64b and other places) that one is not allowed to look at “erva.” There is an assumption therefore that if men are not allowed to look, then women must cover “erva” up. Therefore if a gemara refers to some body part as “erva”, the poskim assume that it must be covered.
Hair covering is unique in that it is D’oraisa and must be covered even if the hair is not considered erva.