Is that it for David Miliband?

The best way I could help Labour was to leave the country. Thus, the man who so nearly became his party's leader explains his decision to head stateside - a decision David Miliband only revealed to close allies in the past two days and didn't dare tell his young children about in case the news leaked out.

I interviewed him in the same room at his north London home we spoke in two and a half years ago after his shock defeat by his brother, Ed. Today I sensed a man who was a prisoner of events beyond his control - who felt unable to take a top frontbench job and yet equally unable to turn one down and stay on the backbenches for fear he would be scripting the next episode of Westminster's favourite soap opera: "The Brothers."

Some, though not him, blame the media for its obsession with that soap opera. In truth, the blame lies with two brothers with two different views of Labour's future and with rival allegiances to Tony Blair and Gordon Brown - the men who used to say that they too were like brothers until they became embittered rivals.

David Miliband clearly believes passionately in the humanitarian work of the International Rescue Committee he's going to head. He is hopeful that his new role will allow him to be involved in international advocacy not just administration. He is cheered by the warm welcome he's had in the USA and the extraordinarily warm words of former President Bill Clinton who has described him as "one of the ablest, most creative public servants of our time".

Equally clearly, though, this is a man resigned to accepting second best rather than someone who is euphoric about starting a new life or making a clean break.

He refused to tell me whether his brother had offered him jobs or whether he'd turned them down - that was me being incorrigible, he said.

Labour can - not necessarily will - win the next election, he says. He refuses to give a list of what the party can do to ensure it does win. Everyone knows what that list would probably have on it - more "restless rethinking" and less "reassuring" the Labour Party, propose don't just oppose, be more candid about past economic mistakes, show how you'll control the deficit, be reformers of the state not simply defenders of it ...etc

Most telling in our interview though was his reaction to Tony Blair's prediction that this was "time out, not time over". He laughs at the words of his mentor and then reinforces them by insisting that "I have not even started this job, let alone thinking about the next."

As he plans to leave these shores David Miliband is reminding the Labour Party what they've rejected and what they lack.

His return to British politics may look unlikely but it is far from impossible as Peter Mandelson and, indeed, Roy Jenkins proved. He could today have killed off any suggestion that he is the Prince over the Water.

But he simply couldn't bring himself to do it.

Here are some key quotes from my interview:

"I feared being a distraction in whatever role I played at Westminster."

"I feel a sense of sadness, because I am British, I love Britain I am passionate about Labour, but I have had to make a choice about where I can make my best contribution."

"Has it been hard for me to accept that I can best help the Labour Party by not just giving the space between the front bench and the back bench to Ed, but the space between the front bench and 3,000 miles away? Yes, but I think that's right."

"I have wrestled with this very, very hard, and I have tried to make a decision that I honestly say to Labour members and supporters that is right for me and for the Labour Party. It is very important that the choice is clear between what Labour is offering to do for the country and what the Tories are doing to the country now."

"I want it to be the vision Ed Miliband has versus the vision David Cameron has. Not Ed and David Miliband. I didn't want to become a distraction, I didn't want the soap opera to take over the real substance of what needs to be done."

On Labour's prospects at the next election, David Miliband said:

"I say what Ed says, the election is winnable, but he is the first to say don't be complacent."

"I came into politics to try to make a difference, and now I am leaving politics to try to make a difference in a different way, on a global scale."

"I am taking a job in America, not taking citizenship in America, I will continue to follow what's going on here, but my focus is going to be making a difference through the International Rescue Committee."

And asked why Ed Miliband should be Prime Minister, he said:

"He will be a great Prime Minister who will do a better job of leading the country towards social justice and economic prosperity and international strength than the current government."