"Prince Charles has said that he wants a slimmed down monarchy but I don’t think he had in mind it would slim down quite as quickly and quite as much as it has,” Phil told the British publication.
He continued: "We’ve got Prince Philip who’s retired, we’ve got the Queen who is 94, Princess Anne is 70 this year. Prince Charles and Camilla are in their 70s.
"Harry and Meghan have left, and Andrew is out in the cold. It is slimmed down a lot already and I’m not sure, if it slims down much more, they’re going to struggle to fit all the jobs,” Phil added.
Royal correspondent Juliet Rieden previously told the Today Show there had been rumours about a much “slimmer” royal family, and she believed Charles’ plan was to head in that direction.
“When Prince Charles becomes King he will slim down the monarchy to a core group, which is largely his own family and their wives and children,” Juliet said at the time.
She said the Queen’s decision to allow Prince Andrew to step out of monarchy work is a sign they are moving towards a much more streamlined royal family.
“I think that is a good move, because the British public does not want to be paying taxpayer money for someone who they see as bringing the monarchy into disrepute,” she added.
But with family members seemingly leaving the royal fold left, right and centre, Phil believes Prince Charles will be forced to abandon his vision for the future of the monarchy.
Despite Charles' slim pickings, Phil went on to say that there is potential for royals such as Sophie Wessex, who is still young, to step up and be more involved in the Firm.
That said, even Sophie has admitted that her children – Lady Louise Windsor, 16, and James, Viscount Severn, 12 – are unlikely to take up official royal titles or duties in the future.
With Louise and James, who are respectively 12th and 13th in line to the throne, not likely to take on duties, it appears the future king will have to look elsewhere to find royal offspring to step up.