Ten years ago, Prince Harry was given the run of Clarence House for his 30th birthday. Guests headed by Prince William and including long-standing childhood friends enjoyed a black-tie dinner where they were reportedly serenaded by Ellie Goulding while Charles and Camilla discreetly decamped to their Scottish hideaway.

Such a family celebration now seems unthinkable as Harry prepares to mark his 40th on Sunday.

Five thousand miles and 10 tumultuous years away, he may ponder, as he did following his dramatic departure from the UK: “What on earth happened? How did we end up here?”

The “here” is the celebrity enclave of Montecito in Santa Barbara county, California, and the “how” – since played out incessantly in newspapers, TV interviews and courtrooms – has been laid bare in his scathing memoir, Spare.

Cast adrift by the hardline “Sandringham summit”, brokered by the royal family, he is seemingly so estranged from William that his brother reportedly does not want him at his own coronation.

So who is Harry as he marks his milestone birthday with a private celebration in Montecito with family and friends followed by a weekend in the mountains with a few of his closest mates?

In his own words, on the sussex.com website, he is a “humanitarian, military veteran, mental health advocate and environmental campaigner”. He is also a “family” man; something he has said he “wanted so badly”.

“It’s amazing. I love every single day,” he said of his US life in February. “The kids are doing great,” he told Good Morning America, adding: “I am just very grateful to be a dad.”

And he is embracing his birthday. “I was anxious about 30, I’m excited about 40,” the duke told the BBC in a statement. “Whatever the age, my mission is to continue showing up and doing good in the world.

“Bring on the next decade.”

According to reports, he is to inherit money from a trust set up by his great-grandmother the late queen mother. William and Harry were to receive £6m when they reached 21, with the bulk going to Harry in the knowledge that William would inherit the Duchy of Cornwall and, later as king, the Duchy of Lancaster. A further £8m was said to have been set aside for the brothers when they turned 40, the Times has reported.

Home, today, is a nine-bedroom, £11m mansion on the Sussexes’ gated 2.2-hectare (5.4-acre) estate, where mononymous neighbours include Oprah and Gwyneth. Hard-won privacy protects Prince Archie, five, and Princess Lilibet, three. Harry is finally liberated from the royal institution he regarded as toxic, and appears to be financially independent.

“Fundamentally, I think he’s happier because he has a family. And I think that’s pretty crucial, actually,” said the royal commentator Peter Hunt, a former BBC royal correspondent.

Yet, sections of the UK media persist in unrelenting negative coverage of a prince they see as having abandoned king and country, and who has, indisputably, made damaging slurs against family and institution.

Was such an outcome inevitable? The stark “in or out” ultimatum delivered at Sandringham – which refused Harry and the Duchess of Sussex a semi-royal role – divined the couple’s path to Montecito, and, presumably, the interviews, documentaries and memoir that followed.

It set a simple narrative: Charles, William and the late Queen Elizabeth were right. “And, therefore, Harry has got his comeuppance,” Hunt said.

“But there is an alternative narrative that argues that, actually, Harry was treated very badly. They could have found a way to accommodate him if they had chosen. There are countless examples where the royal family has adapted, when needed, to survive. But there was too much bad blood by then. They were so entrenched.”

By the time of the infamous summit to determine the couple’s future, a row between William and Harry, including over allegations made, and denied, about Meghan’s treatment of staff, had escalated into the heir allegedly pushing the spare into a dog bowl. Meghan had made Kate – or Kate had made Meghan (recollections may vary) – cry over bridesmaids’ dresses. And Harry, to the royals and the tabloids’ apparent horror, was intent on taking on the media. As was his wife.

He remains committed to his legal battle against some UK media organisations over his claims of unlawful information gathering. “I think he’s utterly determined to see through this legal action for what he sees as media malpractice, which I think could be described as both brave and foolhardy in equal measure,” Hunt said.

So, as he turns 40, Harry remains out the cold in the Californian sunshine.

Anonymous sources have reportedly claimed that a restless Harry is keen for a partial return to the UK, that he’s ditched Hollywood publicists and recently contacted former royal aides and old friends looking for a low-key entry back. Named Operation Bring Harry in from the Cold, the Guardian understands such speculation is wide of the mark, and that, now and for the foreseeable future, he sees his future in California with his family.

Why, ventured one source, would they have spent the past four years establishing their freedom, only to go straight back? They have established business and media interests, including their $100m Netflix deal, along with their Archewell Foundation with its charitable mission of “show up, do good”.

Harry’s circle of friends may have changed. “I’ve lost a few friends in this process,” he has said. Some “chastised” him for the Oprah interviews, he has admitted, others for Spare – the royal family will be relieved to hear that he’s not updating the book for its October paperback release. Yet, it is understood he remains close to a tight circle of old friends – said to include his old school pal Charlie van Straubenzee and Mark Dyer, a close aide to him and William. Both are godparents to Archie.

Loyalties may have been torn on occasion. At the recent society wedding of the Duke of Westminster,who is a close friend to both brothers, William was an usher while Harry was absent, having reportedly declined his invitation out of consideration for the bride and groom.

His new circle reportedly includes friends of Meghan: the tennis player Serena Williams, the actor Katharine McPhee, Meghan’s Suits co-star Abigail Spencer, and the actor and neighbour Priyanka Chopra, to name just a few. Tyler Perry, the actor and film-maker, who lent the couple his Los Angeles mansion when they fled to the US, is godfather to Lilibet. Harry and his neighbour Orlando Bloom text each other, usually updating each other on the activities of the local paparazzi.

In the UK, however, phone calls to his father, with whom he desires rapprochement, often go unanswered, one friend told the Sussex-friendly People magazine. Harry, embroiled in legal action against the Home Office over the axing of his automatic police protection while in the UK, seemingly believes it is something the king can fix, the source said. The issue has “created an impenetrable wall between Harry and Charles”, the friend claimed, with the conversation shifting from frustration to “complete silence” from the king.

And with William, angry at the criticism of Kate in Spare and TV interviews, bridges have not been mended. The two had “barely exchanged a word” by the time of the queen’s 2022 funeral, Harry revealed. At a Norfolk memorial service last month for their uncle, Lord Fellowes – who was married to Lady Jane Fellowes, a sister of Diana, Princess of Wales – both brothers reportedly attended separately, but witnesses did not see them speak.

Apart from attending high-profile awards ceremonies and fundraisers – such as one hosted by their Santa Barbara neighbour Kevin Costner – the couple are occasionally spotted at chic local restaurants. Residents may see a low-key Harry out cycling with Archie, walking home from the gym, or enjoying the nearby beach with the couple’s three rescue dogs. As Vanity Fair’s royal correspondent, Katie Nicholl, recently observed: “I think he’s as close to ordinary as he’s ever going to be in Montecito.”

Harry is understood to be looking to the future with excitement, eager to continue pursuing the work he’s most passionate about, particularly after a year filled with what he believes have been significant accomplishments: expanding the global reach of the Archewell Foundation, visiting Colombia and Nigeria, and celebrating the 10th anniversary of the Invictus Games.

The four-day tour to Colombia promoting causes close to the Sussexes’ hearts – including child online safety – garnered international media coverage. Described as a “quasi-royal tour” by some, the photo spreads it generated served as a reminder that Charles’s vision for a slimmed-down monarchy did not factor in the pulling power of the photogenic Sussexes.

“Ten years ago, we certainly expected him to be a working royal,” said the royal author and historian Hugo Vickers, who predicts problems ahead for the couple. “They will do anything to keep going, I think. They are relying on their royal links, on their celebrity, to keep reinventing themselves and keep themselves in the public eye. And I think it’s going to end badly, personally.

“They may very sincerely believe they are doing wonderful good for the under-privileged of the world. But also, their lifestyle must be incredibly expensive.”

There will have been teething problems as the couple strived to define who they are, and what their purpose is. Perhaps, as they refine their roles, this explains the high turnover of staff. Josh Kettler, Harry’s chief of staff, left on the eve of their Colombia visit after just three months, it having been apparently mutually agreed he was not the “right fit”.

The Sussexes have been getting on with making money to maintain financial independence. However, their five-year Netflix deal expires next year, with no news of whether it will renew. Harry and his Argentinian polo-playing friend Ignacio “Nacho” Figueras have completed a behind-the-scenes polo documentary. Meghan has plans for a lifestyle/cookery series. A $20m Spotify deal ended early, after just one series of Meghan’s Archetypes interviews, though still netted the couple an estimated $12m, along with the epithet “The Fucking Grifters” from Bill Simmons, the head of podcast innovation and monetisation. Spare, a New York Times bestseller, is estimated to have bagged Harry tens of millions of dollars.

Though stripped of his military and royal patronages by the late queen, Harry still globetrots to promote the Invictus Games, the charities Sentebale, Scotty’s Little Soldiers, WellChild and African Parks, the sustainable travel firm Travalyst, and the coaching platform BetterUp.

At 40, Harry may not yet have reached where he thinks he can be. The perfect scenario was to get what he and Meghan originally asked for: to move and still be semi-royals. “So they’re finding another way of doing it,” a friend told the Sunday Times recently.

In Spare, standing in Frogmore House, Windsor, surveying the royal burial ground after the funeral of Prince Philip, Harry briefly dwelt on the fate of his great-great-uncle and former king, Edward VIII, and his bride, Wallis Simpson. “Both of them fretted about their ultimate return,” he mused.

Like Meghan and himself, they too had been driven abroad. Now Edward and Wallis are buried in a remote corner of Frogmore, away from the other royals in “one final exile, maybe”, Harry wrote.

“I wondered how Wallis and Edward felt now about all their fretting. Did any of it matter in the end?”

Is it a question, perhaps, Harry may now be pondering himself?