The source added the Queen is "more isolated" and the royal couple - who celebrated their 72nd anniversary today - is "making an extraordinary personal sacrifice for the country."
What's more, Her Majesty is said to be "under strain" dealing with the fallout of Prince Andrew's Newsnight interview alone, as Prince Charles is on tour in New Zealand.
"The Queen is under strain. She’s telling everyone she’s fine because that’s what she does," a source told The Sun. "But she’s 93 and it’s obvious this is going to take a real toll, so everyone is incredibly concerned.
"A couple of years ago she seemed very content. Charles and Camilla were sorted, William and Kate were doing well and then Harry finding a wife. But now her problems are piling up, both inside and outside the family."
"She’s trying to hold a country together post-Brexit and the Scottish independence referendum," the source continued. "And now the crisis over Prince Andrew follows all the heartache over her grandsons."
"It’s certainly going to be a strange Christmas for her — and does she really deserve this after all she’s done?"
With endless reports of royal rifts, scandals and Prince Philip's declining health, the Queen may be experiencing "Annus Horribilis II."