CDS
I know you can read a Rambam, and probably know that Rambam already. He clearly says what the the problem is, and it exactly what you say makes it kosher.
“avoda zara if not within Hashem’s system… the egel was an outside entity that the yidden came up with”
Here’s the English Rambam from Chabad.org
Their mistake was as follows: They said God created stars and spheres with which to control the world. He placed them on high and treated them with honor, making them servants who minister before Him. Accordingly, it is fitting to praise and glorify them and to treat them with honor. [They perceived] this to be the will of God, blessed be He, that they magnify and honor those whom He magnified and honored, just as a king desires that the servants who stand before him be honored. Indeed, doing so is an expression of honor to the king.
They clearly percieved it within the system. The Ramabm goes on as everyone knows and says only later false prophets took them away from Hashem, they remembered who they ultimately were serving, just they had an intemidiary. An I termediary who was closer to Hashem, who if they honored also honerd Hashem ect..