I don’t know anything about the halacha, but it seems highly bizarre to me to assume that a man is not looking at the women because he is “too busy”.
If anything, he is not doing his job properly.
Also, there is no argument of it being necessary because of the danger, since you should just not go swimming.