The memories the Prince was reffering to was those of his mother Princess Diana who had joined Prince Charles for a six-week tour of Australia and New Zealand back in 1983 with baby William in tow.
Just 21 at the time, Diana dazzled and delighted the crowds. It was her first overseas trip since marrying into the firm and while the public hailed the visit a victory, for Prince Charles it was anything but for he soon realised just how much the public preferred his beautiful young wife to him.
“The prince was embarrassed the crowds so clearly favored her over him,” wrote Sally Bedell Smith in her biography, Prince Charles: The Passions and Paradoxes of an Improbable Life.
“For her part, Diana was upset by the disproportionate interest in her, especially when she realized that it was disturbing Charles. She collapsed under the strain, weeping to her lady-in-waiting and secretly succumbing to bulimia.
“In letters to friends, Charles described his anguish over the impact ‘all this obsessed and crazed attention was having on his wife.’”
Meanwhile, author Andrew Morton that the future king did not like the fact Diana was over shadowing him.
"While Diana looked to her husband for a lead and guidance, the way the press and public reacted to the royal couple merely served to drive a wedge between them. The crowds complained when Prince Charles went over to their side of the street during a walkabout.
WATCH: Princess Diana and Prince Charles take to the dance floor during their royal tour of Australia
In public, Charles accepted the revised status quo with good grace; in private he blamed Diana," Andrew wrote in his biography Diana: Her True Story—In Her Own Words.
“As Diana told me: ‘He was jealous; I understood the jealousy but I couldn’t explain that I didn’t ask for it.'”
Reflecting on the hysteria of the royal tour, Project host Lisa Wilkinson who was editor of Dolly magazine at the time said that it was hard not to be mesmerised by the People's Princess. Almost instantly, the entire country was under her spell.
"I don't think there was any generation that wasn't fascinated by this young girl who we later discovered had pretty much been thrown to the wolves," she told the Sydney Morning Herald in 2017.
Ruthie Farrar, an RAAF flight steward working on the VIP aircraft assigned to the royals at the time also told the publication that it was clear that a then young Diana was still learning the ropes. "Nobody seemed to have told her what to do or trained her," Ruthie said.
The Meghan Effect
Ahead of Harry and Meghan's tour, the Queens former spokesperson Dickie Arbiter told The Mirror that the trip would be a real test for the Duchess.
“She’s a good communicator, she’s good at talking to people but it will be a real baptism of fire.
"They will have a very demanding schedule and the Duchess will have to get used to local customs with the eyes of the world on her. But no doubt she will come through with flying colours and they will show they will be a force of good for the Commonwealth and the Royal Family on the global stage.”
And that's exactly what they did.
If measured by the huge crowds of adoring fans that lined the streets to catch a glimpse of the couple - the tour was a massive success.
Just like Diana 40 years earlier, Meghan charmed with her natural warmth, sense of style and ability to relate to everyday people and children.
"I honest think the Queen and Prince Charles would have been delighted that Meghan was so well recieved on the tour," royal expert Phil Dampier explains.
"The Queen wanted Harry and Meghan to have a massive role representing her throughout the Commonwealth because she can't travel long haul anymore. She saw them as ambassadors."
Phil adds that Her Majesty had absolute confidence in Meghan and knew she would shine.
"She thought Meghan would go down well with different people from all backgrounds and I'm sure she would have done."
At one event in Sydney, it was estimated that a crowd of 15,000 people waited to see the newly weds, who announced on the first day of their trip that Meghan was pregnant.
According to various reports, an audience of that size had not even appeared for Queen Elizabeth on any of her trips Down Under.
"The big difference of course is that Charles started to get envious of the attention his wife was getting and thought he was being sidelined, while Harry appeared to embrace it," Phil continues.
Yet Harry believes it was after this whirlwind tour which saw Megan's global profile soar just like Diana's did, that he noticed a real shift in attitude from his family towards her.
“I just wish that we would all learn from the past,” Harry said when asked by Oprah if he thought particular members of the royal family were jealous.
The Duke went on to describe Meghan as "one of the greatest assets to the Commonwealth" hinting the Royal Family had missed an opportunity by losing the Duchess.
"I mean, here you have one of the greatest assets to the Commonwealth that the family could have ever wished for."
But Phil insists that while Sussexes tour shared similarities to the Prince and Princess of Wales, he does not believe it was the reason behind the couple's decision to step down as senior members of the royal family in January of 2020.
WATCH: Watch the adorable moment between Prince Harry, Duchess Meghan and an Australian toddler
"You could see how people warmed to her on the tour and there were shades of Diana-mania, athough nothing will ever be on that scale again," Phil continues.
"William and Kate might have been slightly anxious that Harry and Meghan would overtake them in the popularity stakes but I don't think that would have lasted. There was room for both the couples and they could have worked together as the Fab Four, not against each other."
Despite this, Princess Diana's former protection officer Ken Wharfe believes there's always been an element of jealousy between Prince William and his younger sibling once dubbed the 'party prince.'
Recalling the brothers upbringing, Ken previously told OK! Magazine that William would get upset if Harry was ever receiving more attention than him.
"William was helpful to his little brother to an extent, but if he saw him getting more attention, he didn't like it.
"I think William often played second fiddle to his brother, simply because Harry was very popular and that was very difficult for him."
Author Anna Pasternak agrees, telling Vanity Fair in April last year that Harry & Meghan's glam-factor during the early stages of their romance made the Cambridges appear "dowdy, surburban, and rather dull."
"That does not go down well in the palace," Anna said.
Meanwhile, speaking to Entertainment Tonight, Battle of Brothers author Robert Lacey said: "William was always sort of jealous of the loveable rogue repuation that Harry had and the affection that he generated.
"And thre is certainly a school of thought that William (and his) wife Kate were not happy at the way in which these royal rock stars, Harry and Meghan, overshadowed them, and they're quite happy now to have seem them out of the country."
Robert continued: "There is a school of thought that it suits William very well for Harry and Meghan to be the scapegoats living abroad."