A man’s upper body is not considered erva like a woman’s. However, tznius would dictate that you should remain modest and not go about undressed where it is inappropriate to do so. Thus, many rabbeim say that you should remain dressed as you would normally, unless engaging in activity that calls upon a particular type of dress that you would normally not don. For example, wearing shorts to play sports and exercise is acceptable, and so is being shirtless when bathing for pleasure.
There’s a different inyan for a person who is a talmid chacham. Kavod haTorah may require such a person to avoid appearing in front of others undressed. This would mean a rebbe should avoid being at a public mikveh when his talmidim are there. I’m pretty positive it’s muttar for a talmid chacham to go to a public mikveh, and appear undressed in front of other men, so you would think that being shirtless while swimming is muttar for a TC when not in front of others. Also, a father shouldn’t appear totally undressed in front of his son at a mikveh, even if the father is not a talmid chacham. However, even when at the mikveh, one should never walk around undressed, just to quickly get in and out of the water.
I don’t know if it’s mandatory or just lifnim meshuras hadin, but my high school rabbeim were makpid to wear long pants and shirts when engaging in water activities with their talmidim such as swimming or rafting. I don’t have the mekoros for all of this, it’s what I was taught by my roshei yeshivahs.