I am new here.
There seems to be evidence that during the shmad in southern Spain, the Rambam’s family had to attend the mosque every Friday (until they left, they were given a year to leave) under the threat of being killed otherwise.
This might explain why the Rambam wrote a letter to his son when he arrived in Akko, saying “nitzalti min hashmad”.
This would be consistent with what the Rambam wrote about “Ya’avor ve’al yehareig” if it isn’t avoda zara.