Sen. Bill Cassidy (R-La.) is trying to explain his vote to confirm prominent anti-vaxxer Robert Kennedy Jr to lead the Department of Health and Human Services as he prepares to leave the Senate.

Cassidy, a physician who chairs the Senate Health, Education, Labor and Pensions (HELP) Committee, provided the crucial deciding vote to advance the nomination of RFK Jr despite his reservations about Kennedy’s anti-vaccine views. Cassidy claimed at the time that he had received commitments that Kennedy would work within established public health guidelines.

In an interview with “Face the Nation” on Sunday, Cassidy told host Margaret Brennan that it would be “easy to surmise” that Kennedy made promises to win his vote.

He added that Kennedy would have been given a role in the Trump administration in some capacity being able to influence public health, and he would rather the prominent antivaxxer be in a position where he would have some guardrails.

“The president seems to be fascinated with the Kennedys, so either he was going to be in a position where there were guardrails, and I did have commitments made as to what kind of guardrails, or he was going to be appointed White House health czar, in which case he would have the president’s ear without the guardrails,” Cassidy said, adding that he determined to vote “to have the one with the guardrails.”

“You can criticize that,” he said, “but I think, again, as I’m searching for, kind of, the right diagnosis and the right cure –– you can criticize me –– but I’m not sure it’s quite as, kind of, black and white as people like to say.”

Cassidy, who recently lost his re-election bid to a Trump-backed candidate in the GOP primary, said Kennedy has broken several promises he made to him during the confirmation process.

One such broken promise is an asterisk on the CDC’s autism and vaccines page, indicating that the page “has not been removed due to an agreement with the chair of the U.S. Senate Health, Education, Labor, and Pensions Committee that it would remain on the CDC website.”

“Once you lose trust in somebody, you’re not quite sure what to trust going forward,” he said. “In fact, you don’t trust anything. It should go away, because the evidence is that that is not the case.”

Yet, Cassidy did not call for Donald Trump to fire Kennedy or for the HHS secretary’s resignation. Instead, Cassidy repeatedly said “[RFK Jr] serves at the pleasure of the president.”

“The commitments that were made to me have been violated. When commitments are made, trust is destroyed. It’s difficult to have effectiveness,” Cassidy said.