Another potential problem is where to build it – the maskana of the gemara is not clear as to if we need a navi or just a mesorah as to where the makom is exactly, and nowadays we don’t have either.
Take a look at where R’ Akiva Eiger, the Chassam Sofer, and many other Achronim have many, many tshuvos about how to practically rebuild the beis hamikdash, and when exactly we are mechuyav.