The imam behind the proposed Islamic center and mosque near ground zero is largely avoiding New York City because of security concerns and is receiving protection from the New York Police Department, according to those close to the imam.

“There’s just a lot of crazies and that’s why he has police protection from the NYPD,” the Rev. James Parks Morton said Monday of his friend Imam Feisal Abdul Rauf.

“I know that he’s being guarded, that he’s not been staying in the usual places, that he’s not working from his office and that he’s concerned for his safety,” said another friend of the imam’s, Rabbi J. Rolando Matalon, in an interview Sunday night.

“He’s staying in an undisclosed location, as they call it,” said Matalon, a rabbi at B’nai Jeshurun on New York’s Upper West Side.

The New York City Police Department did not respond to requests for comment.

Asked whether the FBI was investigating threats against Rauf, New York field office spokesman Bob Margolin declined to comment.

A media representative for Rauf, Seth Faison, would not comment on any security concerns.

Morton, former dean of New York’s Cathedral of St. John the Divine, said the imam was mostly avoiding New York, where he typically works, since returning from a State Department trip to the Middle East two weeks ago.

Morton has known Rauf and his family since the 1960s and works in the same office building as Rauf on Manhattan’s Upper West Side.

Rauf did not attend a Monday morning press conference at Park51, the site of the proposed Islamic center. Zaheer Uddin, Executive Director of the Islamic Leadership Council of New York, said security concerns kept the imam away.

(Read More: CNN)