President Barack Obama is driving a new push to close the Guantanamo Bay detention center despite congressional opposition. A dozen prisoners recently transferred out since Obama vented his frustration to senior administration officials in a rare private meeting.

Under Obama’s watch the prisoner population has been whittled down to 136, from a high near 700 and its lowest point since shortly after it opened in 2002. The reduction is part of Obama’s uphill push to try to fulfil the pledge he made on his first day in office to close the detention center for terrorism suspects on the U.S. Navy base in Cuba.

“We’re working on it,” Obama said at a bookstore over the Thanksgiving weekend when a shopper expressed hope Guantanamo will close.

The sudden surge in transfers during the past months comes after few detainees moved out earlier in the year. Fed up with the stalled progress, Obama summoned administration officials to the White House Situation Room on Nov. 19, delivering a lengthy lecture about why he wants to see it close, according to administration officials familiar with the meeting, which wasn’t on Obama’s public schedule.

Half of the current Guantanamo population — 68 detainees — have been cleared for transfer. Administration officials say at least five more will be moved by the end of the year.

The transfer of some of those detainees in particular is raising objections from House Armed Services Committee Chairman Howard “Buck” McKeon. The California Republican wrote Defense Secretary Chuck Hagel Wednesday saying, “The release of these detainees raises considerable questions and concerns about the risk to Americans.”

The letter could refer to the administration’s effort to send four Afghan detainees back to their home country, where U.S. troops remain deployed amid a resurgent Taliban threat. But McKeon’s office and administration officials would not comment on whether the Pentagon has sent classified notification to Congress that the transfer has been approved, which is required 30 days before a detainee can leave.

Hagel had been reluctant to sign off on guarantees that the Afghans and other detainees would not pose a security threat, drawing White House frustrations. Administration officials said the president’s message in the Nov. 19 White House meeting was primarily directed at Hagel, who resigned five days later under pressure, with Guantanamo one of the issues behind his departure.

Guantanamo opponents say Obama’s nominee to replace Hagel, former Pentagon official Ashton Carter, hasn’t taken a public stance on the prison, but they are hopeful he can overcome reluctance within some parts of the Defense Department to closing it.

The White House would not comment on Obama’s discussions with Carter over Guantanamo. But an administration official, speaking on a condition of anonymity without authorization to go on the record, said Obama wouldn’t pick someone for the job who doesn’t understand closure is a top priority.

Some members of Congress argue that it should remain open to hold and question terrorism suspects. Most of those cleared for transfers are from Yemen, where a violent al-Qaida affiliate makes it too unstable to send them home.

But the day after Obama’s meeting, Georgia and Slovakia accepted the first Yemenis to leave since 2010, proving that resettlement is possible after years of struggle to find countries willing to take them. It will require painstaking diplomatic work to resettle the Yemenis who remain.

Administration officials say more prisoners who previously were classified as too dangerous to be let out are expected to be cleared for transfer in an ongoing review.

But the administration doesn’t want to release some detainees, including the prisoners facing trial by military commission for war crimes — a group that includes five men charged with planning and aiding the Sept. 11, 2001, attacks. Their fate remains the final stumbling block to closure.

Closing Guantanamo would still either require approval from Congress, which has prohibited transferring any prisoners to the U.S., or a bold unilateral action that Obama’s opponents are warning him against but administration officials say hasn’t been ruled out.

(AP)