Life as a bug hunter

Bug hunting coders often compete to track down programming errors

Vulnerabilities or bugs are errors made when computer programmers write their code.

They happen for the same reasons journalists make typographical errors in copy: weariness, inexperience and on very rare occasions malice.

In the olden days of programming, code-reviews amongst departmental peers were a common way to catch these flaws before they hit production systems, much like a sub-editor on a national newspaper might proof-read a correspondent's copy before it goes to the printers.

But these days, tighter budgets and intense pressure to get a product ready by its release date means code isn't checked so thoroughly and bug frequency rises.

In pre-internet times this wasn't too serious a problem -- it could be contained. If your code had errors then it might bring down a mainframe order processing system or perhaps an airline's reservation desk would be out-of-sorts for an afternoon.

Nowadays a bug in a software product can mean massive security breaches for customers and financial and reputational disaster for a business.

It is something Bill Gates came to understand somewhat belatedly. In a now famous missive he sent every full-time employee at Microsoft an e-mail in 2002 emphasising a fundamental shift in priority:

"Now, when we face a choice between adding features and resolving security issues, we need to choose security," he wrote.

If you are being charitable to the software manufacturers then this emphasis on security in recent years is a diligent response to an explosion in online fraud.

A more critical view has it that the good guys always seem to be playing catch up to the baddies.

Cash for code

That is where the bug bounty hunters come in.

Mozilla - makers of the Firefox web browser - were first to start a bug bounty programme in 2004. Their top prize is currently $3,000 (£1,800) and they have paid out about $40,000 (£25,000) per year since then. Their top earner is a student in Germany who has bagged more than $30,000 (£18,000) from a series of discoveries.

This year Russian programmer Sergey Glazunov became the first person to claim Google's highest bounty of $3133.70 (£2,000) for finding a weakness in its Chrome browser.

Brian Rukowski, product manager for Chrome, said the company had paid out over $50,000 (£31,000) in bug bounties so far. The reward usually depends on the severity of the bug found, and for Google the amounts are honed to attract a specific type of person.

Google has cash rewards to bug hunters to help refine its Chrome web browser

"When we started out it was $1337 which if you write it down spells out 'leet' which is hacker speak for elite. Since then we've increased the top prize to 3133.70 which spells 'eleet,'" explained Rukowski.

As well as gaining some cash and kudos it is also an ideal way for programmers to showcase their skills. For the really bright, the prize could even be a job offer.

Chris Hofmann, director of special projects at Mozilla, said: "We are constantly looking to hire and contract participants that get involved in the security bug bounty program."

It is a view echoed by Brian Rukowksi from Google: "We have found a couple of people who, either through reporting bugs or just by working on the source itself, look very promising. We've recruited them."

Magic

The 25-year-old Aaron Portnoy has been tracking down bugs since he was barely into his teens. He first realised it could be a potentially lucrative career when tax collectors from the American Internal Revenue Service came calling, wondering how he had made $60,000(£37,000). Aaron was not even 20 at the time.

He describes the feeling of bug hunting as an "exhilarating experience. It's as if we are magicians of the computer".

Although he managed to earn some income, Portnoy felt that the majority of security researchers weren't getting the recognition they deserved from the software makers:

"Vendors would simply add a 'Thank You' to their patch. Over the course of the last four or five years we've seen a lot of researchers not feeling that's enough. Giving them a small thank you, a pittance, it's just not enough for them," he said.

Spotting a gap in the market, Portnoy turned this dissatisfaction into a business opportunity and in 2006 he joined Tipping Point software and the team he manages has a simple aim.

"We were looking for a way not only to create a business model but also to reward those researchers for their work."

How the model works:

Instead of notifying a software company directly, a bug finder contacts Tipping Point with their discovery.

Portnoy's team investigates and if the flaw is verified, the researcher is paid a fee.

The manufacturer is notified and hopefully starts working on a permanent solution. In the meantime a temporary fix is created by Tipping Point and released to its paying subscribers.

Tipping Point is now part of the Hewlett-Packard company and under this scheme 'Platinum' bounty hunters can achieve single payments of $20,000 (£12,500).

These days Portnoy helps run the annual Pwn2own competition.

At the event, programmers compete to find flaws in popular software and hardware products. This year Irish programmer Stephen Fewer earned $15,000 (£9,000) when he 'took down' Microsoft's Internet Explorer.

For the manufacturers, Pwn2own is a win-win situation. It is a great way to prove their products are bulletproof if they survive the onslaught. On the other hand, if their product is breached then they have the advantage of finding out about a major flaw in a controlled environment.

That is why companies like Google actively participate, according to Brian Rukowski.

"In the last couple of years Chrome has fared very well. We've had the distinction of being the only browser left standing at the end of the competition so we've got a bit of a target painted on our back now and we're excited about that."