How to make the web work in real-time

The web brings the Battle of Trafalgar to your fingertips

When you do pretty much anything on the web it involves your browser asking for data from another source.

Refresh a webpage and your browser will go away and ask if there is a updated version you should be seeing.

This gentlemanly approach works well most of the time. Want to find out when the Battle of Trafalgar was fought? No problem. Ask a search engine and it will tell you because it checked earlier.

However, its shortcomings are becoming apparent as the online audience swells.

That ask-and-you-will-receive mechanism breaks down when too many people want data from a server at the same time. Everyone knows the frustration of websites that will not respond because they are overwhelmed.

A change is under way to help the web cope with this growth and help some parts of it cope better in a real-time age.

The change comes in the form of software specifications known as the Extensible Messaging and Presence Protocol (XMPP).

Despite the complicated name, XMPP does a very simple thing. Whereas the web, via HTTP, asks for information on behalf of an individual, XMPP makes sure everyone is told.

If HTTP is a performance for one, XMPP is a stadium gig.

All for one

The technology has been around for a few years but is now starting to be used in many more ways. Once widely deployed it will sit alongside the more familiar HTTP and be put to very different uses.

"HTTP is the way you query information that is in databases," said Mickael Remond, a member of the XMPP standards group and head of instant messaging firm Process One. "XMPP is a real-time channel for publishing events."

Some airlines are using XMPP to help high-value customers.

German airline Lufthansa has turned to an XMPP-based service for its high-value customers. The technology lies behind an instant messaging application that it provides to its most frequent flyers.

"They can use it all around the world where ever they are to ask a question," said Stephanie Renda, head of tech firm Match2Blue which built the messaging application.

"These mobile users expect to receive information at the right time and in the right place," said Ms Renda. "They do not want to search."

"The information must find the customer not the other way around," she said. Lufthansa keeps an eye on the stream of messages passing through to spot potential problems - such as when a volcanic ash cloud grounds all flights.

The technology also underlies an application developed by veteran New York nightclub Webster Hall that allows its patrons to find like-minded folk to chat to at the sprawling venue.

Google's Wave collaboration tool is based on XMPP, social networking giant Facebook is starting to use it and increasing numbers of instant and social messaging tools are being built around it.

"There's a transition going on," said Dr Giles Nelson from Progress Software which uses XMPP to spot trends in streams of events. "XMPP is about pushing the information out rather than waiting for the people to come and ask."

"That's going to become more and more important on the web generally," he said. "The information sources you want, the data will be pushed to you and will become available immediately."

By using XMPP, Progress Software can instantly find out about very complicated changes in the information it is monitoring.

"XMPP is going to be there underneath helping with complex event processing," said Dr Nelson. "Analytics on those feeds could reveal time-based patterns such as so many tweets on this subject in a certain period of time."

Search futures

XMPP is also going to be key to the future of what most of us do, most of the time online - search.

XMPP helps Webster Hall patrons find friends at the sprawling night spot

Google has become one of the biggest firms interested in XMPP because it knows that when the web is totally real-time, the way it does search now will break down.

Rather than Google have to go and find all the information online by crawling websites it would prefer to have the information come to it.

Also, said Pontus Kristiansson, head of behavioural marketing firm Avail Intelligence, something like XMPP is going to be essential as we move from a web of pages to a web of people.

"As the volume of data increases and the speed with which data is updated accelerates, a centralised process to try to determine relevant information for individuals will be far too slow, no matter how you engineer it," he said.

When the web is all about what people are doing, what they are saying right now, then a different way of handling that data is required.

"The way you declare you are interested in something is simply by behaving like you do," he said. Instead of people having to report what they are doing to a page so others can see it, the information might be published via XMPP to all those who are interested.

Those people might not be interested in five minutes time, but they are now so getting that timely data to them is imperative.

XMPP and the move to a more social web could change the face of the network, he believes.

"The information flow will become so real-time that traditional search will be obsolete," he said.