Michael Cohen, President Donald Trump‘s former lawyer and fixer, was returned to federal prison on Thursday after refusing to sign a document that would have blocked him from speaking to the press or publishing a book during the remainder of his sentence.

Cohen, who plead guilty in 2018 to campaign finance violations including paying hush money during Trump’s 2016 campaign to a former adult film actress and a former Playboy model who claimed that had affairs with Trump.

Cohen was scheduled to complete his sentence in November 2021, and had been staying at home on a medical furlough since May 20 due to the coronavirus spreading in prison.

He had expected to serve the rest of his sentence under home confinement after the furlough, but was surprised by the no-media agreement.

Cohen’s legal adviser, Lanny Davis, said that Cohen believed the documents violated his First Amendment rights. However, once it became clear he would not be allowed to stay at home without signing off on the agreement, Cohen relented.

However, according to Davis, three federal marshals took Cohen back into custody, even though he had agreed to sign the documents at this point.

The Times reported that the document contained eight conditions, including “no engagement of any kind with the media, including print, TV, film, books, or any other form of media/news.” The purpose of the rule was to “avoid glamorizing or bringing publicity to your status as a sentenced inmate serving a custodial term in the community.”

In a statement, the federal Bureau of Prisons said that Cohen had been returned to custody after he “refused the conditions of his home confinement.”

The news comes a week after Cohen announced on Twitter that he planned to publish a book in late September.

The “ruling” Cohen was referring to was that of a New York state judge who reversed a lower court decision, allowing the president’s niece, Mary Trump, to publish her tell-all book about Donald Trump.

Davis confirmed to the Times that the book was ready for publication and would reveal Cohen’s experiences working for Trump.