On Thursday, Ronny Jackson withdrew from consideration to become the next Secretary of Veterans Affairs in the wake of allegations that he routinely got intoxicated while on duty and over-prescribed medication.

Ronny Jackson Pulls Out As VA Secretary Nominee

The White House physician was also accused Wednesday of drunkenly driving and crashing a government vehicle years prior during a farewell party for a Secret Service employee. Jackson, a former rear Admiral in the Navy, has denied all claims against him.

“The allegations against me are completely false and fabricated,” Jackson said in a statement, according to Politico. “If they had any merit, I would not have been selected, promoted and entrusted to serve in such a sensitive and important role as physician to three presidents over the past 12 years.”

Jackson joined the White House when former President Barack Obama hired him in 2013. The accusations against the doctor, which also include the creation of a generally hostile work environment, date back to 2015, before President Donald Trump took office.

SLIDESHOW: DONALD TRUMP’S 30 CRAZIEST TWEETS

Even before the allegations against Jackson began surfacing, many lawmakers began questioning whether he was qualified to lead a department as large as the VA in terms of pure experience. This week, however, several legislators from both major parties denounced Trump’s nominee for the position. Some even revealed that Jackson was nicknamed the “candy man” because he often handed out large doses of prescription drugs like Ambien and Percocet as if they were sweets.

Trump chose Jackson to replace David Shulkinwho was ousted as VA Secretary after reportedly disagreeing strongly with several administration officials on multiple important issues.

The president vehemently defended Jackson after the allegations against him arose, saying in an interview on Fox News on Thursday morning that the claims were simply an attempt to “destroy” the doctor’s reputation. Jackson issued Trump’s health report in a press conference and January, and drew criticism for overly praising the president’s body despite him being borderline obese.

Trump also criticized Jon Tester, a ranking member of the Veterans’ Affairs Committee, after the Democratic Senator blasted Jackson’s behavior.

“I want to tell you that Jon Tester, I think this is going to cause him a lot of problems in his state,” Trump said. Tester is up for re-election in Montana in November’s midterms, and the race is projected to be a big challenge for the Democrat, as Trump easily won the state in 2016.

Trump was asked by Fox News on Thursday whether he had any ideas for potential replacements for Jackson, and the president said he had thought of someone but did not reveal who.

“I do, actually. Better not give it,” said Trump. “I think we’ll have somebody great. Somebody that is more — look, the admiral is not a politician, which is what I liked. … Somebody with political capability, yes.”