Even back in 1948 - the year after Elizabeth and Philip married, and while she was pregnant with Charles - rumours surfaced that Philip's eye had wandered and fallen on musical comedy star Pat Kirkwood. She was the girlfriend of society photographer Baron Nahum, a member of the infamous Soho Thursday Club, which Philip was also known to frequent.
In 1957, the Baltimore Sun reported that Philip was said to have been interested in a party girl - causing a rift with Elizabeth - and speculation was that the girl in question was Pat. The story resulted in an unprecedented statement to the press authorised by Elizabeth to quench the gossip: "It is quite untrue that there is any rift between the Queen and the Duke."
So did Prince Philip have an affair with Pat?
Pat has recounted that she did spend one memorable night out dancing with Philip - and her boyfriend - with them all ending up eating scrambled eggs for breakfast. And her fourth husband has said he has correspondence between Pat and Philip that will prove there was no extramarital affair.
Well this could have made for awkward family dinners.
Philip was rumoured to have had an affair with Sarah Barrantes, the mother of Sarah Ferguson - aka Fergie, Duchess of York, who would go on to marry Prince Andrew, the middle son of Philip and Elizabeth, in 1986.
Nicholas Davies reports in his 2000 book Elizabeth II: Behind Palace Doors that Susan and Philip became acquainted in the late 50s/early 60s due to their mutual connection with polo - in fact, there is some truth to the lure of horsing around, but perhaps not with Philip: Susan would end up leaving Fergie's father, Major Ronald Ferguson, for Argentinian polo player Hector Barrantes.
A childhood friend of the prince - he gave her away at her first wedding when she was 20 and he was only 16 - Hélène, who became a TV personality, was infamous for having had two children out of wedlock, Max (b. 1943) and Louise (b. 1945).
With Philip stepping up to become their godfather and even said to have chipped in for school fees for Max to attend Gordonstoun, where he would send his sons Charles, Andrew and Edward, rumours spread that Hélène's children - born before his wedding to Elizabeth - must have been fathered by the prince.
In 1996, Max issued a statement saying that Philip, who has remained close with the family over the decades, was not his biological dad.
Countess Mountbatten: Penelope Brabourne
Penny is married to Norton Knatchbull - the eighth Baron Brabourne - who is a godson of Prince Philip. After her husband left her in 2010 for an affair, she became even closer with both the Queen and Philip, who she has known since she was introduced to Charles in the 70s, and has remained so, even after Norton returned to the estate.
"Her nickname among Palace staff is “and also” as no guest list is considered complete without her," says Express.co.uk.
But her frequent visits to the royal residences have sparked rumours that Penny, a butcher's daughter, is more than just carriage-driving partners with Philip. The claims are reported in the 2018 book The Queen's Marriage by Lady Colin Campbell.
As the Queen is said to be an admirer of Penelope, who is now known as Countess Mountbatten, it is unlikely there is much truth to the rumours of scandal between her husband and her friend.
Rumours of an affair with Sacha, the Duchess of Abercorn, started in 1987 when the duke was photographed in a towel with his arm around the duchess, who is 25 years his junior.
"Our friendship was very close [but] I did not go to bed with him," she told author Gyles Brandreth, as reported in his 2006 book, Philip and Elizabeth: Portrait of a Royal Marriage.
Other Duke of Edinburgh affairs
Among the Prince's many other alleged liaisons include rumours of dalliances with the actress Merle Oberon, the novelist Daphne du Maurier, and even his cousin Princess Alexandra.
"There is no doubt in my mind at all, The Duke of Edinburgh has had affairs — yes, full-blown affairs and more than one," biographer Sarah Bradford has said.
"And the Queen accepts it. I think she thinks that’s how men are."
Regardless of whether he has cheated on the Queen or not, after four children and the better part of a century together, Philip and Elizabeth seem to have the marriage thing down pat, as testified by Elizabeth in her 1997 golden wedding anniversary speech: "He has, quite simply, been my strength and stay all these years, and I, and his whole family and this and many other countries, owe him a debt greater than he would ever claim, or we shall ever know."
Aww, shucks. Now that's a right royal declaration of love.