Big data unites the study of stars with cancer research

The Milky Way can be analysed by algorithms

The study of the stars and the fight against cancer may seem to have little in common but the two have been brought together by the algorithms that read big data.

Every day we create 2.5 quintillion bytes of data - 90% of the data in the world today has been created in the past two years alone.

This amount of data has in turn driven a revolution in ever smarter computers that can make sense of all that information.

Medicine is just one area benefiting from this closer relationship between information and machines.

The collaboration between astronomers and oncologists began at a cross-disciplinary meeting in Cambridge to discuss data management.

At the meeting, Dr Nicholas Walton, an astronomer, met James Brenton from the Cancer Research UK Cambridge Institute.

Big data algorithms are advancing the study of cancer

Prof Carlos Caldas, who also works at Cancer Research UK, explains what happened next.

"Astronomers are looking at pictures of the sky, but they can't sift through millions of pictures by hand, so they use imaging algorithms that can analyse and classify objects," he says.

"We obtain images from humans. Could we deploy the same algorithms to read that data?"

The answer was yes and the algorithm has been advancing the study of cancer ever since.

"Using the astronomy algorithm, we can automatically classify hundreds of thousands of cells, we can look at patterns, how cells are related to each other, we can precisely count them and find the average distance between cells," Prof Caldas says.

"It speeds up diagnosis and allows us to glean information that had previously been glossed over.

"It is completely transforming pathology into the digital realm.

"The sky is the limit."

Scientists at Cancer Research UK have just announced a breakthrough in the way they "read" breast cancer cells.

It means that they are now able to create a 3D map that links the shape of breast cancer cells to genes turning on and off, matching it to real disease outcomes.

It is not just the diagnosis of disease that can be transformed by big-data insights.

"Data is exploding but so is the technology and that understand and exploit it," said Nick Millman, a managing director at consultancy firm Accenture.

"In health, the techniques that have previously been used in marketing analytics that have allowed brands to understand an individual's preferences can be applied to wellness - how to encourage someone to follow a healthier diet, for instance."

What can a Facebook "like" say about you?

Big data is the key theme of an exhibition currently running at London's Science Museum, which looks not just at how it is transforming industry but also how our own data footprints are affecting our privacy.

Sheldon Paquin, the exhibition's curator, told the BBC: "This silent revolution has completely changed everything about our lives, from how we look to the stars, to our trips to the doctor, to how we talk with one another. We imagine the modern world to be fast-paced, connected and interwoven with technology, all things that we owe inescapably to big data.

"We can now examine the evolution of language, search the genome for disease, navigate cities yet to be built and identify our common fears. Our increased connectivity is making us easier to read."

The exhibition features a study done at Cambridge University, which illustrates how easy it is to read our personalities via Facebook likes.

Volunteers were asked to complete a series of personality tests and that was compared with their Facebook data.

"We found we could predict personality as accurately as a spouse from the average person's Facebook likes," said David Stillwell.

"We could predict intelligence, personality traits, age, gender, religious views."

Predictive World invites users to see what cities might know about them

Now the team has turned its attention to working out how much data we share with our increasingly smart cities - in a partnership with games publisher Ubisoft to promote its futuristic city game Watch Dogs 2.

Just as Watch Dogs, which is based in a city run by an operating system that collects and analyses data on every citizen, Predictive World assumes that privacy will be a distant concern in our future urban environment.

The tool generates psychological predictions from users' Facebook data as well as finding relationships between gender and salary, location and crime risks, personality and life expectancy.

But can citizens turn their data footprints to their own advantage?

CitizenMe is one of a growing number of firms that aims to offer people the chance to better understand the data that is collected on them as well as allowing them to sell it.

It enables people to collect copies of their data from around the internet and put it into an app on their phone.

"These insights and data quickly become very valuable. If they wish to, citizens can choose to anonymously exchange some data for immediate cash reward," explained StJohn Deakins, the founder of the company.

The app is currently being trialled with about 10,000 people.

How much data?

• 350,000 tweets every minute

• 400 hours of video uploaded to YouTube every minute

• three million Facebook posts uploaded every minute

• four million Facebook "likes" every minute

• four million Google searches every minute

• 2.5 quintillion bytes of data are now circulating around the world, 90% of which has been created in the past two years

• a total of 44 zettabytes of data will be in circulation by 2020