10 Downing Street has recently confirmed that Boris Johnson, the current Prime Minister of the United Kingdom, will be issued a fixed penalty notice for breaching COVID-19 related laws in June 2020.

Several commentators have suggested that it is very unusual, perhaps even uniquely so, for a sitting Prime Minister to be found to have broken the law. For example, the BBC says:

Mr Johnson's fine makes him the UK's first serving prime minister to be sanctioned for breaking the law.

whereas The New York Times makes the slightly different claim that:

making him (Johnson) the first holder of his office to be found breaking the law in living memory.

Are these claims correct? Have any other British prime ministers been found to have broken the law, while in office? I don't know of any counter-example, but I am still somewhat surprised by this. Have there, perhaps, been cases where a prime minister resigned shortly before being found to have broken the law? Regarding the New York Times' statement, were there prime ministers predating living memory who were found to have broken the law, while serving or otherwise?