Gen. Mark Milley, the Chairman of the Joint Chiefs of Staff, apologized for joining President Donald Trump at St. John’s Bible photo-op, admitting it was “a mistake” that he “learned from.”

Gen. Milley released a pre-recorded speech Thursday, stating he regretted walking over to Lafayette Square with Trump and appearing at St. John’s Church for Trump’s Bible photo-op while wearing combat uniform.

“I should not have been there. My presence in that moment and in that environment created a perception of the military involved in domestic politics,” said Milley. “We must hold dear the principle of an apolitical military that is so deeply rooted in the very essence of our republic.”

The photo-op took place after police officers forcefully cleared out Lafayette Square of peaceful George Floyd protestors with rubber bullets and tear gas so that Trump and the military officials could walk from White House to St. John Church. The scenes of the tear-gassed crowd, recorded by reporters and protestors, sparked public outrage.

Sen. Lindsey Graham (R-South Carolina), released a statement, supporting Gen. Milley’s video statement.

“I have nothing but deep admiration for and total confidence in General Milley as Chairman of the Joint Chiefs of Staff,” Graham wrote. “I support his statement in both substance and spirit regarding the recent presidential visit to St. John’s.”

Defense Secretary Mark Esper, who was also present at the photo-op, did not release a statement, though he commented on the situation last week at a press conference at Pentagon. He told reporters he “was not aware of a photo-op was happening.”

Trump repeatedly called in for tougher measures to suppress the demonstrators, criticizing local governors for their way of handling protests. Trump threatened he would to deploy active military force on protesting civilians.

Despite the presence of 1,600 troops in Washington D.C., Pentagon did not deploy them. Esper told reporters at a press conference he did not regard it as necessary.

“The option to use active-duty forces in a law enforcement role should only be used as a matter of last resort, and only in the most urgent and dire of situations. We are not in one of those situations now. I do not support invoking the Insurrection Act,” he told reporters.

After the statement, the Wall Street Journal reported Esper had a tense exchange with Trump after a disagreement on the use of military troops. Several anonymous officials told the Journal, Esper contemplated resigning.