Too Big to Know: Rethinking Knowledge Now That the Facts Aren't the Facts, Experts are Everwhere, and the Smartest Person in the Room is the Room [Tapa dura]

The premise of this book is that somehow networked organizations and networked thinking will lead to better, smarter decisions. As long as we include a sufficient diversity of opinions and experience in the networks helping us make our decisions we will arrive at better, more informed answers. In fact, as the amount of information explodes, these networks will be the only way to manage all the information we are creating.

Here's the problem. I don't think anyone will dispute that reaching out the to internet to search for knowledge can get reasonable answers quickly. Also, running contests where many experts are involved can get good results. The problem is, if you are solving a real problem at the end of the day somebody actually has to do the work to get an answer. A "network" isn't going to magically come up with an answer. Also, reaching out to a wide group on the internet often results in the same stupid *wrong* answers to a problem being circulated around and around and around. Networks can just as easily work in a negative direction recycling stupidity rather than knowledge. There doesn't seem to be much of a role in this book for sustained critical and deep thinking about a problem to arrive at a solution. This doesn't make sense to me since much of human progress continues to come from sustained hard work by individuals working to achieve expertise in an area and focusing on a single problem at a time. This book makes some good points about how our relationship with information is changing to rely more on networks of our colleagues or friends to filter and absorb the massive amounts of information created every year. However, the author's confidence that networked thinking and organizations will magically solve many of our problems is happy nonsense, in my opinion. Also, while the author claims that networks can make better decisions, he never really gives any detailed research supporting this assertion or showing under what conditions networks are better or worse at solving problems. This makes the book more of an exercise in faith rather than something you can use to decide if a network would be helpful in a particular problem or not.

Having his background in philosophy, perhaps no one had expected David Weinberger to write a book on a topic that is at the heart and soul of librarians, i.e. cataloguing and classification. In the modern notion of the term this is called metadata. When Everything is Miscellaneous was published in May 2007, at first it was as if some war was waged against Melville Dewey's classification system, especially the class 200 for Religion. Some protagonists in the field such as Peter Morville responded with an apt blog entry arguing that "Not Everything is Miscellaneous". In his book, even more in his several book talks, Weinberger mocked not only Melville Dewey and Michael Gorman but also Aristotle, albeit with a great caution. In many ways though, the book has slowly been well received and cited widely in the library and information science literature. The book would be considered as disruptive in its argument against some of the conceptual foundations of library and information science, mainly classification and categorisation systems. In Everything is Miscellaneous, Weinberger called for a total rethink of not only the notion of classification systems but also the very definition of metadata. For him, "metadata is what you already know and data is what you're trying to find out" (Weinberger, 2007, p.104).

Now his new book is out as of early January 2012. Too Big To Know: Rethinking Knowledge Now That the Facts Aren't the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room. In this book, Weinberger offers yet his staunchest critique on well established conceptual and theoretical foundations of knowledge including the DIKW (Data- Information-Knowledge) pyramid that most computer science and library science have incorporated in their curriculum in their foundations course.Another concept he took aim is information overload. A typical Google search on the phrase information overload returns more than 6 million results (doubled even since Weinberger records this statistics). Popularised by the technology futurist Alvin Toffler, the phrase resonates in the minds of librarians who for so long have hinged their value proposition on solving the problem of having too-much-information. As Weinberger notes, information overload, also called info glut, data smog, or information tsunami, is a problem so serious that it has become a topic for a whole body of work. Not only that, the problem also warranted inclusion into the scientific and psychiatric dictionary with its nomenclature such as information anxiety or information fatigue syndrome. In Too Big To know, in what seems a disruptive argument, Weinberger tells that too much information is actually a good thing. To support his argument, he cites Clay Shirky, who argues that "it is not information overload. It is filter failure".

By providing several examples and writing rather beautifully, Weinberger contrasts the long-form argument of the Age of Books with the loosely connected webs of the Age of Networks in which he argues, the long form argument is a constraint inherited from the medium of print. Our thought process, nonetheless, works not in a simplistic, linear and long form ways but in an intricate web of links and associations which is better reflected in the Age of Networks. Scientists work in private in the Age of Books, after-the-fact peer-review is the norm, but in the Age of Networks, he argues, the filtering process is immediate, open and on the cloud. In short, he argues the abundance of crap and good that is generated through the network gets filtered by the network itself.

Reading this book, one can surmise that Weinberger is for Open Access. He is for Open Internet. He is for Open Data. He is for Linked Data. Such an open ecology, Weinberger argues, provides a fertile ground for innovation and creativity. Overall, influenced by less baggage from the disciplines of either computer science or library science, Weinberger seems to suggest that the influence of the Age of Books is fading and the time has come for the Age of Networks. Hence, he argues, knowledge is now residing in the network, not on any one or even genius skull.

In many respects, Weinberger's Too Big To Know, is in agreement with arguments put forward by James Surowiecki's The Wisdom of Crowds and Clay Shirky's Here Comes Everybody. Perhaps a slight oversight in the book may be the notion of `information overload as good and inevitable' was first discussed by David Allen in his book Getting Things Done (2002). We will await reflections from other authors such as Andrew Keen who may argue against some if not most of the views espoused by Weinberger.

The book is relevant to the field of library and information science in many ways. The discussion on the value of metadata, especially on metadata that is generated by users in terms of user tagging, ratings, reviews, filtering, and recommendations (pp.186) is crucial for librarians who are at the cross-roads of choosing between old and new metadata paradigms. This book illuminates new ideas on library collections and provides a glimpse of what the future of libraries would look like, albeit Weinberger's discussion is at a philosophical level. Metadata is ultimately one of the solutions to the filter failure in which he asserts "the solution to the information overload problem is to create more metadata". This is a notion, I believe, we in library and information science should develop further.

A few reviewers complain that this book is a bit scattered and lacking a clear, explicitly stated, thesis. I somewhat agree with them, so let me try, as best I can, to give you what I think the author's core message is.

In the vein of philosophical postmodernism, David Weinberger's underlying idea is that what we come to call knowledge changes in 'shape' with the media we are using to convey and absorb it. And for quite a long time, we have been using the media of the printed word, in scrolls, books, magazines, academic journals, etc. And the consequence of this is that knowledge appears to be somewhat neat, tidy, and resting on foundations. Thus, I go to a library to get a book (which is published only after a rigorous peer review or editing process), and read its very linear argument where one chapter builds on another to reach a conclusion. When needed, the author cites authorities in footnotes, which I will seldom check myself because of the time and energy (if not monetary) cost involved. And while the author can anticipate my objections, we are having a one-way conversation where the author is talking to me (and where I can talk to myself in an 'inner dialogue' but not to the author).

Now, Weinberger writes, since the technology is changing, how we think about what knowledge will surely change also. First, it is becoming glaringly apparent how little information any one of us can absorb. While information was abundant with books, this fact was somewhat 'hidden' because only a fraction of all total information was published, and only a fraction of that was carried in libraries and bookstores, and only a fraction of that was ever seen by any individual reader. Now, of course, we are painfully aware of how much info there is because the internet makes it cheap to publish and brings it to us (rather than expecting us to go to it).

In addition, instead of looking at citations I seldom expend the time/energy to verify, things are becoming hyperlinked. And with hyperlinking comes a challenge to the linear ways of the book - where one followed the argument where the author, rather than the reader, wanted to take it. (How many times have you got lost following hyperlinks, versus how many times have you got lost in checking physical citations?)

Etc.

So, maybe, the world where information appears scarce, long-form books are the pinnacle of what knowledge looks like, and where knowledge rests on firm foundations (citations which MAY but seldom are checked let alone challenged) is a thing of the past. On the internet, anything can be published relatively cost-free, discussion boards and blog posts + comments yield as much knowledge as (and more rapidly than) books, and everything is hyperlinked, including hyperlinks (the citation process ends up looking less like a straight line and more circular).

So what?

Well, Weingberger offers some possible suggestions for how to cope with this (hypothesized) change, but he leaves those for a very short last chapter (of which I will not divulge details). The other reviewers are correct to note that this book is less focused than it could be, and because of that, readers very probably WILL come away with their "so what?" not having been answered. Weinberger is content to make his argument that the future of knowledge will probably change and leave it at that. (Of course, along the way, he disagrees vehemently and convincingly with those who think "the internet is making us stoopid" by, in essence, pointing out that our idea of what "smart" is was largely shaped by the technology we use, the book being the supposed measuring stick for what intellectual achievement is. And that begs the question that is at issue.)

So, this really isn't a 'how to' book, or a book that will be of much interest to business folk (or others) looking for tips on how to deal with informational trends. It is much more a book of philosophy whose interest lies in... its just being so interesting. The author's case is really novel and while I think his exuberance may lead to some overstatements (there was no such thing as basing theory on fact until Bacon? Ever heard of Galileo?), most of these areas don't, in my view, affect his overall case. The irony is that it takes a long form book - published by a publishing house, replete with footnotes rather than hyperlinks - to make it.