Monday 15 April 2013

Book review: Big Data, Viktor Mayer-Schonberger and Kenneth Cukier (2013)

The age of Big Data started years ago and we all need to manage it. Just yesterday, in Sydney, there was a story about a university using students' online activity to predict performance, with administrators saying they'll step in to help if needed. The use of large amounts of data to predict outcomes is having an affect within companies and governments, and within society generally, across the world, and it's predicated on the low cost of data processing and storage.

The book contains many examples of Big Data activity, mainly in the US. It also contains warnings, as the use of massive quantities of data enables organisations to forsee the future through the use of correlation rather than causation. The authors say that we have to be careful not to allow mathematics to negate the notion of free will, on which our legal system is based. Just because it's likely that a person might commit an offense, doesn't mean that he or she should be arrested because a computer algorithm has decided that this is so.

Mayer-Schonberger and Cukier even propose the creation of a new profession of "algorithmists", men and women who can understand the complex mathematical tools that are being developed to handle Big Data, and use their understanding to make sure that evil is not committed. The two men also touch on the issue of data storage horizons. How long should data about people be kept when the likelihood is that new, useful applications of data might only be discovered years or decades hence? Most people already think about this issue in relation to their lives online, and about the right to be forgotten. As data manipulation for civic and commercial purposes becomes more commonplace such issues will arise in the public sphere, and governments will be forced to tackle them even through legislation.

The two writers have produced an interesting and engaging book, although the first 30 pages or so might be cut; there's a certain bureaucratic necessity in them that disappears once talk turns to how Big Data is actually used, later in the book. They turn to Big Data projects of the past, as well, sometimes retracing our steps to the fairly distant past.

The men are certain that the world will change in significant ways once Big Data becomes more common. This book is an introduction to new possibilities, and a warning to new forms of abuse that were not cogent before the era of Big Data emerged.

No comments: