Huberman on Communities of Practices and Forecasting

Bernardo Huberman, the director of the Information Dynamics Lab at Hewlett-Packard, gave a talk at Stanford on January 8 entitled, “Information Dynamics in the Networked World.” Huberman covered two somewhat disparate topics: Automatically discovering Communities Of Practice by analyzing email Social Networks, and a general forecasting technique based on markets.    (TX)

The first part of his talk was a summary of his recent papers. I’ve alluded to this work here many times, mostly in reference to Josh Tyler and SHOCK. The premise of the work is similar to that posed by David Gilmour in his recent Harvard Business Review article.    (TY)

Huberman described a technique for discovering clusters of social networks within an organization by analyzing email. The key innovation is an algorithm for discovering these clusters in linear time. The algorithm, inspired by circuit analysis, treats edges between nodes as resistors. By solving Kirchoff’s equations (which gives “voltage” values for each node), the algorithm determines to which cluster a node belongs.    (TZ)

The second part of Huberman’s talk was enthralling: Predicting the near-term future using markets. This is not a new idea. A very topical example of a similar effort is the Iowa Electronic Markets for predicting the outcome of presidential elections.    (U0)

The methodology Huberman described (developed by Kay-Yut Chen and, Leslie Fine) aggregates the predictions of a small group of individuals. It works in two stages. The first provides behavioral information about the participants, specifically their risk aversion. Huberman remarked that risk aversion is like a fingerprint; an individual’s level of risk aversion is generally constant. In the second stage, participants make predictions using small amounts of real money. The bets are anonymous. Those predictions are adjusted based on risk aversion levels, then aggregated.    (U1)

Huberman’s group set up a toy experiment to test the methodology. There were several marbles in an urn, and each marble was one of ten different colors. Participants were allowed to observe random draws from the urn, and then were asked to bet on the likelihood that a certain color would be drawn. In other words, they were guessing the breakdown of colors in the urn.    (U2)

Although some individuals did a good job of figuring out the general shape of the distribution curve, none came close to predicting the actual distribution. However, the aggregated prediction was almost perfect.    (U3)

The group then tried the methodology on a real-life scenario: predicting HP’s quarterly revenues. They identified a set of managers who were involved in the official forecast, and asked them to make bets. The official forecast was significantly higher than the actual numbers for that quarter. The aggregated prediction, however, was right on target. Huberman noted that the anonymity of the bets was probably the reason for the discrepancy.    (U4)

The discussion afterwards was lively. Not surprisingly, someone asked about the Policy Analysis Market, the much maligned (and subsequently axed) brainchild of John Poindexter’s Information Awareness Office. Huberman responded that the proposal was flawed, not surprisingly suggesting that this technique would have been the right way to implement the market. It was also poorly framed and, because of the vagaries of politics, will most likely never be considered in any form for the foreseeable future.    (U5)

I see many applications for this technique, and I wonder whether the Institute for the Future and similar organizations have explored it.    (U6)

As an aside, I love how multidisciplinary the work at HP’s Information Discovery Lab is. I’ve been following the lab for about a year now, and am consistently impressed by the quality and scope of the research there.    (U7)

“Low-Focus Thought” in Knowledge Management Systems

David Gelertner wrote an essay called “The Logic of Dreams” (a chapter in Denning and Metcalfe’s Beyond Calculation: The Next Fifty Years of Computing), where he discussed the creative process. Gelertner suggested that there are two kinds of thought: high-focus (analytical, logical) and low-focus (free association). The former we understand well (according to the Gelertner); the latter we barely comprehend.    (TI)

Low-focus thought is the story of weak ties, not just in the context of Social Networks but of ideas in general. It’s a story that is told over and over again. A poet smells a rose, and is reminded of his lover. Friedrich Kekule dreams about a snake biting its tail, wakes up, and solves the structure of benzene. Grace Hopper remembers an old play from her college basketball days and figures out a memory-efficient algorithm for her A-0 compiler.    (TJ)

Gelertner was interested in implementing low-focus thought in Artificial Intelligence software. I’m interested in facilitating low-focus thought via Knowledge Management systems.    (TK)

In the past year, my tools and processes have revealed a number of unexpected connections. For example, last August, I blogged two interesting articles about Marc Smith and Josh Tyler. The following morning, I happened to be rifling through some old articles, and discovered papers written by Smith and Tyler that I had previously archived.    (TL)

Old-fashioned tools and a little bit of karma led to these discoveries. I wanted to eliminate one of the stacks of papers on my floor, which was how I accidentally came across the Smith article. Later that morning, I was searching for an email that a friend had sent me earlier, and it just so happened that the same email contained the reference to Tyler’s paper.    (TM)

These discoveries were largely due to luck, although the fact that I keep archives in the first place and that I review them on occasion also played a role. I don’t want to oversell this point, but I don’t want to undersell it either. Many people don’t archive their email, for example. Many groups don’t archive their mailing lists, a phenomenon that baffles me. More importantly, many people never review their old notes or archives, which is about the same as not keeping them in the first place. All of that knowledge is, for all intents and purposes, lost.    (TN)

Good Knowledge Management tools facilitates the discovery of these weak connections, and make us less reliant on luck. Blogging is great, because it encourages people to link, which encourages bloggers to search through old entries — both of others and their own. This is an example of tools facilitating a pattern, and it’s one reason why blogs are a powerful Knowledge Management tool.    (TO)

I’m excited about the work we’ve done integrating blogs and Wikis using Backlinks and WikiWords, because I believe these tools will further facilitate low-focus thought, which will ultimately lead to bigger and better things.    (TP)

Knowledge Management as Information Brokering

David Gilmour, CEO of Tacit Knowledge Systems, wrote an excellent (and short) essay in the October issue of Harvard Business Review entitled, “How to Fix Knowledge Management.” The gist of the article:    (P3)

The problem is that most organized corporate information sharing is based on a failed paradigm: publishing. In the publishing model, someone collects information from employees, organizes it, advertises its availability, and sits back to see what happens. But because employees quickly create vast amounts of information, attempts to fully capture it are frustrated every time. Even the most organized efforts collect just a fraction of what people know, and by the time this limited knowledge is published, it’s often obsolete. The expensive process is time consuming, and it doesn’t scale well. (16)    (P4)

Gilmour’s solution:    (P5)

Instead of squelching people’s natural desire to control information, companies should exploit it. They should stop trying to extract knowledge from employees; they should instead leave knowledge where it is and create opportunities for sharing by making knowledge easy for others to find. This requires a shift away from knowledge management based on a publishing model, and a focus on collaboration management based on a brokering model. (17)    (P6)

Tacit Knowledge Systems‘s system does this by scanning all of the email and other documents on a corporate network, and building profiles of individuals based on these behaviors. The system can then alert people to other individuals with similar interests, brokering an introduction between them. If you think there are potential privacy problems here, you’re not alone. Josh Tyler‘s SHOCK works in a similar way, but distributes control of the profile to the individual; see his paper, “SHOCK: Communicating with Computational Messages and Automatic Private Profiles.”    (P7)