David Miliband says he won't join brother Ed's team

David Miliband has announced he needs to "recharge his batteries" away from frontline politics and will not serve in his brother Ed's shadow cabinet.

Mr Miliband, who lost the Labour leadership election by just over 1%, told the BBC the decision was the right one for him, his family and for Labour.

He said he would stay on as an MP and continue to "serve" the party, and did not rule out a return at some stage.

Ed Miliband said "his door was always open" should David want to return.

He told reporters that it would have been "fantastic" if David had chosen to serve in the shadow cabinet - saying it would have made it a "stronger team" - but that he understood the reasons for his "thoughtful and gracious" decision.

But the Conservatives said the development "spoke volumes" about the direction Ed Miliband was taking Labour.

Speculation ends

Speculation has surrounded the former foreign secretary's future intentions since Saturday's leadership election result - which came as a shock to the long-time favourite to succeed Gordon Brown.

He had to decide before 1700 BST whether to put his name forward for elections to the shadow cabinet, which will decide his brother's top team as Labour regroups after losing power in May.

In an interview with the BBC's political editor Nick Robinson, Mr Miliband said he was "absolutely certain" that the decision not to stand was the correct one.

He said it had been his instinct to make the move immediately after Saturday's defeat but wanted three or four days to dwell on the decision and talk it through with his family.

"It is the right thing for me now and certainly for the foreseeable future to support Ed from the backbenches," he said.

By stepping back from front-line politics, Mr Miliband said he would be able to spend more time with his family - a "nice bonus" of losing the leadership contest - and think about his future direction.

'No distractions'

He said his brother needed a "clean field" as he embarks on his leadership and "the freedom and space to drive the party forward in the way he sees fit without any distractions".

But he said he was absolutely determined to remain "engaged with the big issues" facing the party and the country.

In a letter to his constituency party chairman, Mr Miliband said if he remained part of his brother's shadow ministerial team, he feared "perpetual, distracting and destructive attempts to find division where there is none and splits where they don't exist".

He said media coverage of his reaction to Ed Miliband's conference speech - in which he appeared angered by his brother's criticism of Labour's support for the Iraq war - showed the intense level of scrutiny he would be under if he stayed in Labour's top team.

Mr Miliband told the BBC that he would not be involved in "any public spats" with his brother from the backbenches.

"If I have anything to say to Ed, I will be saying it privately," he said.

David Miliband would not be drawn on whether his brother urged him to stay in the shadow cabinet or whether his exit marked the end of New Labour - with which he is closely identified.

But Nick Robinson said there were political as well as personal reasons behind his departure stemming from the nature of his leadership defeat - which saw him win the most support from both MPs and party members but trail behind his brother in terms of the votes of union members.

'Relieved man'

Ed Miliband said Britain "hasn't heard the last" of his brother and that he would continue to make a "big contribution to politics" whatever he chose to do.

Describing him as a "massive talent", he said he "looked forward to the possibility" of David Miliband returning to the front bench either in opposition or government.

Many colleagues of David Miliband's had urged him to stay and help Labour take the fight to the coalition government, but senior figures praised his decision for putting the interests of the party first.

Veteran Labour frontbencher Jack Straw said he believed David Miliband would be a "relieved man" as his decision would spare the party the "complications and soap operas" that would result from the two brothers serving side by side.

Shadow home secretary Alan Johnson said it was the "right decision" for both brothers and for the party, saying the media would have scrutinised "every nuance of policy difference" between the two, when they actually agreed on most things.

Former Labour leader Lord Kinnock said he regretted the decision as David Miliband was "needed in front-line politics" but said he "understood and respected" the reasons for it.

The Conservatives reacted to his exit by saying it signalled that Labour was "vacating" the centre ground of politics.

"David Miliband was a leading architect of New Labour," said party chairman Baroness Warsi. "The fact that he doesn't want a place in Ed Miliband's shadow cabinet speaks volumes about the direction in which the new leader is taking Labour."

In his absence, 49 Labour MPs have put their names forward for the shadow cabinet elections. The candidates, including a host of former ministers and several without ministerial experience, are competing for 19 seats around the table.