Soaking the Menorah in very hot water with some Arm & Hammer washing soda (not to be confused with baking soda) works great if it’s not aluminum (then it’ll corrode). If it’s silver, I think the black engravings will come off as well. So I just use this for the kids brass menorahs, but it should work for glass or other metal etc. as well.
To clarify RD’s tip, you preheat the oven THEN TURN IT OFF before you put the menorah inside. I don’t think the paper towels are necessary, just in case someone mistakenly turns the oven on.