There is no secondary meaning. When you call someone who has low IQ “retarded,” you don’t also mean “insult.”
No, the insult has to do with how the person is not properly using his intelligence, whereas retarded refers to an inherent lack of intelligence.