Lord Geidt said he repeatedly told the PM’s team to be ready to explain if his actions stuck within the rules – even if he thought there was no breach.
But he said the advice had not been “heeded”, calling on Mr Johnson to set out his case to the public.
The PM said being fined by the police does not break the ministerial code.
Writing to Lord Geidt, Mr Johnson said he had “no intent to break the regulations”, and that he had been “fully accountable to Parliament and the British people”.
The ministerial code outlines the rules government ministers must follow when in office, including the “overarching duty” on them to comply with the law.
If the code is broken, the convention in Westminster is for a minister to resign.
Labour’s deputy leader, Angela Rayner, said the report was “the latest sign of the rampant sleaze engulfing Downing Street”, adding: “This prime minister has been found out and his days are numbered.”
Liberal Democrat MP Wendy Chamberlain also said it showed the PM’s own ethics adviser “no longer trusts him to tell the truth”, and that Mr Johnson was “not fit to hold public office”.
The Metropolitan Police carried out an investigation into lockdown breaking parties in Downing Street and Whitehall after allegations surfaced towards the end of 2021.
A total of 126 fines were handed out to 83 people as a result of the force’s investigation into 12 events during the pandemic.
And a wider report by senior civil servant Sue Gray revealed a drunken party culture within No 10 while the rest of the country was ordered to stay at home.
Mr Johnson – along with his wife Carrie and Chancellor Rishi Sunak – was given a fixed penalty notice by the Met for attending a birthday party thrown in his honour in June 2020.
Following Ms Gray’s report last week, the PM apologised to the Commons, saying he had been “humbled by the whole experience” and had learned lessons.
But pressure has continued to grow, with at least 12 MPs calling on him to resign since the report’s publication.
‘A failure of communication’
Analysis by BBC political correspondent David Wallace Lockhart
Despite some critical language in Lord Geidt’s report, the Cabinet Office is stressing that he’s not resigning.
But a Downing Street source would not deny reports – published in The Times – that Lord Geidt threatened to quit on Tuesday unless Boris Johnson publicly explained his conduct.
The prime minister did publish a letter, in which he denied breaching the code, insisting there was no intent to break the rules.
He went on to stress that he’d apologised.
Lord Geidt also reveals that he repeatedly advised the prime minister – via his officials – to address this very issue in the wake of his fine.
He adds he was assured this was conveyed to Boris Johnson. In his reply Mr Johnson says there may have been “a failure of communication” between their offices.
This is the latest development raising questions about Boris Johnson’s conduct, at a time when almost 30 of his own MPs have publicly called on him to go.
In his annual report on ministers’ interests, Lord Geidt said questions around Mr Johnson’s behaviour had led to an “impression… the prime minister may be unwilling to have his own conduct judged against” the ministerial code.
The standards adviser said it would be “especially difficult to inspire that trust in the ministerial code if any prime minister, whose code it is, declines to refer to it”.
He said when it came to the Partygate fine, “a legitimate question has arisen as to whether those facts alone might have constituted a breach of the overarching duty within the ministerial code of complying with the law”.
And even if the PM thought there was no breach, he should “should respond accordingly, setting out his case in public.”
Lord Geidt – who reports to the prime minister – said he had avoided “offering advice” to Mr Johnson about the PM’s “obligations under his own ministerial code” as he would be forced to resign if Mr Johnson rejected the advice.
But after repeatedly contacting his team “to ensure that the prime minister should publicly be seen to take responsibility for his own conduct under his own ministerial code”, the prime minister had “made not a single public reference” to it.
Mr Johnson said he was “not aware of the weight… put on the absence of an explicit reference to the ministerial code”.
But in his letter to his adviser, he insisted he had not broken it by being fined.
The PM wrote: “I have taken full responsibility for everything that took place on my watch, and reiterate my apology to the House and to the whole country…
“I have also been clear that there was no intent to break the regulations”, he added, before saying: “Paying a fixed penalty notice is not a criminal conviction.”
Mr Johnson also said that, in his view, the same principles applied to the chancellor.