President Donald Trump‘s administration announced Wednesday that some of the 2,000 troops fighting terrorist group ISIS in Syria will be sent home, and GOP lawmakers quickly condemned the White House’s move.

Hawkish Republicans like South Carolina Sen. Lindsey Graham blasted Trump’s decision in a meeting with Vice President Mike PenceAccording to PoliticoGraham told Pence he felt “personally offended” to learn about the decision in the news instead of via the president or his aides.

“We’re going to hold the administration accountable for this decision,” Graham said he told Pence. “If Obama had done this, all of us would be going nuts because it’s such a bad idea. The job of the Congress is to hold the executive branch accountable. He’s the commander in chief, but he needs to be held accountable for his decisions. And I want hearings as soon as possible.”

Graham has wavered between criticizing and praising Trump since he first ran for president in 2016.

In a tweet on Thursday, Trump wrote he found it “so hard to believe that Lindsey Graham would be against saving soldier lives and billions of $$$.”

SLIDESHOW: DONALD TRUMP’S 30 CRAZIEST TWEETS

Graham also responded to a tweet from Trump by saying media outlets are right in reporting that Russia, Iran and Syrian leader Bashar Al-Assad are pleased with the U.S.’s decision to withrdraw from the Middle Eastern nation.

The White House claimed Wednesday that the U.S. and its allies had defeated the Islamic State in Syria (ISIS).

Florida Sen. Marco Rubio was another Republican who warned that the administration’s latest military decision could have major consequences.

“Virtually everyone who spoke, spoke in opposition to the decision,” Rubio said. “I didn’t hear anybody that disagreed with the view I’ve taken … this is a major mistake. And I hope they reverse it. Because if not, I think it will haunt this administration.“

Sen. John Cornyn of Texas, the Senate majority whip, said the White House’s announcement took him and many of his fellow GOP lawmakers completely by surprise. National Security Adviser John Bolton reportedly called Cornyn to explain the move.

Meanwhile, Nebraska GOP Sen. Ben Sasse — another frequent Trump critic whose name has been floated as a potential 2020 presidential candidate — also slammed the decision, noting how many groups would be hurt by it.

“The losers are Israel, humanitarian victims, and U.S. intelligence gathering,” said Sasse of the decision. “A lot of American allies will be slaughtered if this retreat is implemented.”

Not all Republicans were displeased with Trump’s announcement, however. Sen. Rand Paul of Kentucky lauded the move, according to POLITICO. However, Paul did not attend his fellow lawmakers’ lunch with Pence.

In a statement, White House Press Secretary Sarah Huckabee Sanders said these victories over ISIS in Syria do not signal the end of the Global Coalition or its campaign.” Sanders also said American troops will keep striving “to deny radical Islamist terrorists territory, funding, support, and any means of infiltrating our borders.”