Go in. If it’s crowded, just say hamakom, and then go to where the female family members are (even if they’re not sitting, they are usually there – maybe in the kitchen, etc.)
If it’s not crowded, then stay and speak to the avel for a bit.
I was faced with exactly that issue recently. two brothers were sitting shiva for a parent, at the home of one of the brothers. My husband and I are friends with the OTHER brother and his wife. It would have made absolutely NO sense for me to sit and talk with the female members of the household who are not friends of mine, and to be honest, making such small talk seems inappropriate to me in a shiva house. I grant that is was an awkward shiva, but the mitzvah is to pay a shiva call respectfully, and my awkwardness would be of little consequence, if my presence is comforting to the aveil who IS my friend.It happened that my husband and son came with me, but it did not really alter the dynamics of the visit. I say just go, do the mitzvah, and be supportive of your neighbor.