It was Professor Neil Ferguson's advice that prompted Boris Johnson to put Britain in lockdown.
He sat on the government's SAGE committee, which provides advice to ministers on their response to the pandemic.
He also sat on the New and Emerging Respiratory Virus Threats Advisory Group (NERVTAG).
The Government Office for Science confirmed he had 'stepped back' from both bodies.
Professor Ferguson, nicknamed "Professor Lockdown" is an infectious disease epidemiologist at Imperial College London, said he believed he was immune to the virus, after self isolating with symptoms in March.
He leads the team that produced shock research claiming more than 500,000 people would die from the virus in the UK unless lockdown restrictions were introduced.
It's claimed his lover travelled across London on two occasions to spend time with the expert.
He told the Telegraph: "I accept I made an error of judgement and took the wrong course of action. I have therefore stepped back from my involvement in SAGE [the government's Scientific Advisory Group for Emergencies].
“I acted in the belief that I was immune, having tested positive for coronavirus and completely isolated myself for almost two weeks after developing symptoms."
He added: “I deeply regret any undermining of the clear messages around the continued need for social distancing to control this devastating epidemic.
“The Government guidance is unequivocal, and is there to protect all of us.”
According to the Telegraph, Mr Ferguson was visited twice by Antonia Staats, a campaigns manager for US-based non-profit organisation Avaaz.
Ms Staats, 38, is thought to be married, but in an open relationship.
She lives with her husband and their two children in a £1.9m South London home.
