What John Boehner's ambition has to do with the shutdown

John Boehner, left with Obama, is not seen as a conservative ideologue, yet he is following the policy agenda of his most ideological members

On a very simple level, the closing of most of the American federal government can be pinned on the ambitions of one man, Speaker of the House John Boehner.

There is little doubt that Mr Boehner, a Republican, could, whenever he wants, gather enough votes from moderate Republicans and most, if not all, Democrats to reopen the government.

But conventional wisdom says the Republican caucus would swiftly dethrone Mr Boehner. And, interestingly, conventional wisdom and punditry, in Washington at least, don't blame Mr Boehner one bit for holding the government hostage to his career aspirations. Thus is the cynicism of this town today.

Why he is so attached to sticking with such a bruising job is another question. His caucus is balkanised and unruly. This Congress is held in the lowest public esteem since the invention of public opinion polls. And the Republicans are more disliked than the Democrats.

Once seen as a skilled dealmaker, Mr Boehner isn't able to make deals. Never seen as an ideologue, he has now relinquished control of the Republican policy agenda to the party's most ideological faction.

Newt Gingrich (left) led a government shutdown in the 1990s that eventually proved disastrous to Republicans

Where's the fun? To put the question another way: Why can't the leader of the party in Congress control the party on the most important issues and votes?

The answers apply equally to Democrats and Republicans. A series of self-inflicted errors by the two political parties over the past 40 years have left party leaders with no whip and little power.

Half-way reform

After Vietnam and Watergate, there was a reform spirit that wanted to open and democratise the process of selecting party candidates for office, as well as get special-interest money out of politics.

The first part worked too well. Party candidates all came to be picked through open primary elections. In the process, the parties lost the ability to select loyal candidates in smoke-filled back rooms - they lost a source of power and persuasion.

The campaign finance reforms, however, backfired entirely. The post-Richard Nixon idea was to stop party bosses from doling out money from local moguls, unions and corporations. Instead, the reforms deformed and opened the spigots for money to flow directly to candidates from all the old sources, bypassing the party machines. The quantities of money have grown to gargantuan proportions.

By the 1980s, politicians were essentially free agents. They didn't need the parties to get nominated or to fund campaigns. Pollsters, advertising wizards and fundraisers replaced the party bosses. And the party leaders in Congress lost their leverage.

Incumbent power

At the same time, computers and marketing data gave political professionals new precision in drawing the map of congressional districts. Individual districts have become much more homogenous - overwhelmingly Democratic or Republican. It has never been easy to unseat an incumbent. But now, once candidates of the dominant party in these gerrymandered districts get the nomination, they are home free.

So members of the Congress, especially in the House, mostly have safe seats and are immune from challenges by the other party. Their bigger challenges come from within their own parties and that tends to drive them further right or left. Voting patterns are as partisan now as at any time since after the Civil War.

All that means the House is filled with increasingly ideological members who aren't especially worried about re-election and are impervious to the party whip and discipline.

It is legislative chaos.

John Boehner carries an extra burden as a Republican.

From 1949 to 1995, the Democrats controlled the House except for two years in the 1950s. That is a long time to be in the political wilderness, with the Republicans effectively shut out of governing and legislative power. They developed a sort of frustrated minority party mentality, locked out of power and able only to toss bombs, make mischief and obstruct.

Since the 1920s, Republicans have controlled the White House, Senate and House of Representatives at the same time only during parts of George W Bush's two terms. So modern House Republicans have had virtually no experience actually sharing the responsibility of governing. Their minority mentality lingers. And it shows.

Republicans were led back to power in the House in 1995 by one of the most mischievous, incendiary party leaders ever, Newt Gingrich. He led the way to a government shutdown that year that was considered disastrous to Republicans.

But the Gingrich confrontation now seems far more rational than the Boehner shutdown. First off, back then Republicans also controlled the Senate and so had more leverage with the Democratic White House.

And Mr Gingrich's Republicans were pushing for a broad set of coherent conservative changes to the budget. There were realistic grounds for negotiation.

Mr Boehner's Republicans are pushing to repeal a sitting president's hallmark achievement a year after he was re-elected. There is no room for negotiation and no chance of success. It is purely a stunt, an act of guerrilla theatre.

Feud not shutdown

Mr Boehner was part of the House Republican leadership under Mr Gingrich. He was ousted in 1998, survived and was resurrected as party leader years later. One might have thought he had learned some strategic lessons from all that.

It would have been great sport to watch if Mr Boehner had decided to gamble this time and take on so-called "wacko bird" conservatives. If he put together a coalition of more mainstream Republicans and Democrats, there would have been a classic intra-party feud, but there would not have been a shutdown.

It isn't written in stone that Mr Boehner would have lost his speakership after that battle. Perhaps a victory could have shifted the paralytic partisanship in the House today.

Ironically, chances are high Mr Boehner will have to cave eventually and team up with Democrats to avoid an economically irresponsible - and politically lethal - default on government debt obligations.

That will probably save his job, but it certainly won't make his job any more fun.