For the record I believe Popa is right that the Tzedukim did not reject all of the Oral Torah (in the second grade sense). I don’t have the sources off-hand, but they have machlokisim in the Gemara with Chazal regarding dinim that are only found within the Oral Law, where the Tzedukim are more strict than Chazal were, or they reject only part of the din.
Tzedukim put on teffilin that looked like our teffilin. They were not like the later Karaim who read the Torah literally.
Rashi does say that they rejected Torah Sh’Bal Peh. I would suggest that Rashi was either being lav davka (i.e. the reject part of Torah Sh’Bal Peh) or, more likely, he means that they rejected the Authority of Torah Sh’Bal Peh.