A Happy Information Hygiene Moment (and a Great Explanation of the Backfire Effect)

Yesterday, my sister shared this Oatmeal comic that wonderfully explains the backfire effect, the phenomenon where seeing evidence that contradicts our beliefs hardens those beliefs rather than changes our minds.

I love The Oatmeal for its engaging and often humorous visual explanations of important concepts. (XKCD and Nicky Case are also brilliant at this.) My sister knows this, and asked me if I had seen it before. Even though I loved this one, it didn’t ring a bell.

So I did what I try to do in situations like this. Rather than just file it away in my Evernote (where I have thousands of clippings that I almost never see again), I went to record it on the human perception page under “Confirmation Bias” on the Faster Than 20 wiki. To my delight, I found that I not only had seen it before, but I had already captured it on my wiki!

It’s a practice I call good information hygiene (a term coined by my colleague, Chris Dent). When we do it well, we’re not just filing things away where we can find them, we are continually synthesizing what we’re consuming. The act of integrating it into a larger knowledge repository is not only good information hygiene, but is also a critical part of sensemaking. Doing it once is great, but doing it multiple times (as Case and my colleague, Catherine Madden, have also explained beautifully) makes it more likely to stick.

Here’s another, simpler example that doesn’t involve a wiki and may feel more accessible to folks tool-wise. In my late 20s, I met Tony Christopher through my mentor, Doug Engelbart. We had such a great conversation, when I got home, I wanted to make sure to enter his contact information immediately into my contact database. When I opened it, to my surprise, he was already in there! I had very briefly met him at an event a few years earlier, and I had recorded a note saying how much I had enjoyed that short interaction.

I love when moments like this happen, because it shows that my tools and processes are making me smarter, and it motivates me to stay disciplined. I wish that tool developers today focused more on supporting these kinds of behaviors rather than encouraging more fleeting engagement with information.

Misguided Adventures in Stale Knowledge Repositories

Greg Bloom, the founder of the Open Referral project, recently published a project Year in Review. He opened by succinctly telling the story of Open Referral:

We started Open Referral to address a systemic problem: it’s hard to find trustworthy information about the health, human, and social services available for people in need. We have many sources of community resource directory information — with more emerging all the time — but they all struggle to sustain themselves while trying to meet the diverse needs of their communities, and competing with each other (by default or by design) along the way.

We know that it shouldn’t be technologically hard to solve this problem by enabling community resource information systems to cooperate with each other — but it will take a movement to make it happen.

Not only is this a very clear description of the problem Open Referral is trying to solve, this is a very clear description of a problem that many communities have.

Many communities of practice that I work with have a simpler, but no less nefarious version of this same problem. Sharing resources and articles seems like high value, low-hanging fruit. This often takes the form of sharing information in-flow — on mailing lists, Slack, whatever the group’s primary messaging system might be.

This generally works well enough. Contributing and consuming are both easy, and there’s no cost to the information going stale, because you’re using a channel that’s essentially disposable. Sometimes, the channels even include searchable archives, which give you some measure of persistence.

At some point, someone almost always has the bright idea to compile a repository of resources on the web. This almost never works well. People are rarely thoughtful or — more importantly — iterative about figuring out how to contribute, curate, and consume these resources. And so these projects fail.

There aren’t good turnkey solutions for this. The reason is that it’s a hard, context-dependent problem. Good solutions are possible, but you have to commit to being iterative and experimental. If you’re not already iterative and experimental, you can get there through commitment and practice. Doing so is, ultimately, a social challenge, not a technical one.

Greg is grappling with a harder problem, but he’s doing it the right way — by treating it as a social problem, and incorporating technology in a thoughtful way. It’s really cool to see a lot of hard work start to pay off.

Community Engagement and Dynamic Knowledge Repositories

One of the benefits of working on the Hyperscope is not just working closely with Doug Engelbart, but also with many of his long-time colleagues, not the least of whom includes Christina Engelbart, his daughter and former business partner. Christina and I recently had an off-the-cuff exchange on community engagement and maintaining a Dynamic Knowledge Repository (DKR), and it’s started to evolve into a full-fledged discussion. Rather than continue it over email, I thought I’d post some of my thoughts here, especially since they relate to my recent post on Leave A Trail and stigmergy.    (KDW)

On the surface, maintaining a Dynamic Knowledge Repository and community engagement would seem to be two separate actions. Christina suggests that the former is a more tangible entity, whereas the latter is a process. I agree with this distinction, and I would also note that the artifacts of community engagement are part of a DKR. So you’ve got the engagement itself (process), and you’ve got the artifacts of the engagement (part of a DKR).    (KDX)

The distinction starts to blur when the engagement occurs over a digital medium. If I exchange email with someone, then is that community engagement or is it part of maintaining a Dynamic Knowledge Repository? It’s both, because the act of engagement also results in an artifact!    (KDY)

Why is this important? Because when we think of the two as distinct actions, then doing both means double the work. When we think of the two as having significant overlap (via relatively minor shifts in work practice), then we get (at least) twice the benefit for half as much work.    (KDZ)

This conflation is central to methodologies like Dialogue Mapping. Jeff Conklin talks about the importance of Shared Display as being part of the conversation. When the screen is physically part of the conversation, then participants engage with the screen as if it were another participant. That creates shared ownership over the artifact, which makes the artifact far more valuable than a meeting summary that some guy in the corner scribbled onto his laptop and emailed out afterwards.    (KE0)

The benefits multiply even more when you take into account Leave A Trail and scale. Having a closed DKR for a small team is valuable, but opening that DKR up for the entire world to see increases the potential for serendipity and emergence.    (KE1)

I would argue very strongly for folks to think about community engagement and maintaining a knowledge repository as part of the same bucket, because they should be very closely tied to each other. For example, when I add content or refactor the public Hyperscope Wiki, I consider that part of my community engagement strategy. That said, these two concepts are not identical. This is important to remember also. Simply dumping information into a public repository is not a very effective community engagement strategy. However, proactively reaching out to people and encouraging them to interact in the public repository is a great community engagement strategy.    (KE2)

Jeff’s framing is probably the best way to think about it: Your knowledge repository should be thought of as a participant in your collaborative process, not just something external.    (KE3)

OHS Launch Community: Experimenting with Ontologies

My review of The Semantic Web resulted in some very interesting comments. In particular, Danny Ayers challenged my point about focusing on human-understandable ontologies rather than machine-understandable ones:    (5D)

But…”I think it would be significantly cheaper and more valuable to develop better ways of expressing human-understandable ontologies”. I agree with your underlying point here, but think it’s just the kind of the Semantic Web technologies can help with. The model used is basically very human-friendly – just saying stuff about things, using (triple) statements.    (5E)

Two years ago, I set out to test this very claim by creating an ad-hoc community — the OHS Launch Community — and by making the creation of a shared ontology one of our primary goals. I’ll describe that experience here, and will address the comments in more detail later. (For now, see Jay Fienberg’s blog entry, “Semantic web 2003: not unlike TRS-80 in the 1970’s.” Jay makes a point that I want to echo in a later post.)    (5F)

“Ontologies?!”    (5G)

I first got involved with Doug Engelbart‘s Open Hyperdocument System (OHS) project in April 2000. For the next six months, a small group of committed volunteers met weekly with Doug to spec out the project and develop strategy.    (5H)

While some great things emerged from our efforts, we ultimately failed. There were a lot of reasons for that failure — I could write a book on this topic — but one of the most important reasons was that we never developed Shared Language.    (5I)

We had all brought our own world-views to the project, and — more problematically — we used the same terms differently. We did not have to agree on a single world-view — on the contrary, that would have hurt the collaborative process. However, we did need to be aware of each other’s world-views, even if we disagreed with them, and we needed to develop a Shared Language that would enable us to communicate more effectively.    (5J)

I think many people understand this intuitively. I was lucky enough to have it thrown in my face, thanks to the efforts of Jack Park and Howard Liu. At the time, Jack and Howard worked at Vertical Net with Leo Obrst, one of the authors of The Semantic Web. Howard, in fact, was an Ontologist. That was his actual job title! I had taken enough philosophy in college to know what an ontology was in that context, but somehow, I didn’t think that had any relevance to Howard’s job at Vertical Net.    (5K)

At our meetings, Jack kept saying we needed to work out an ontology. Very few of us knew what he meant, and both Jack and Howard did a very poor job of explaining what an ontology was. I mention this not to dis Jack and Howard — both of whom I like and respect very much — but to make a point about the entire ontology community. In general, I’ve found that ontologists are very poor at explaining what an ontology is. This is somewhat ironic, given that ontologies are supposed to clarify meaning in ways that a simple glossary can not.    (5L)

Doug himself made this same point in his usual ridiculously lucid manner. He often asked, “How does the ontology community use ontologies?” If ontologies were so crucial to effective collaboration, then surely the ontology community used ontologies when collaborating with each other. Sadly, nobody ever answered his question.    (5M)

OHS Launch Community    (5N)

At some point, something clicked. I finally understood what ontologies (in an information sciences context) were, and I realized that developing a shared ontology was an absolute prerequisite for collaboration to take place. Every successful communities of practice had developed a shared ontology, whether they were aware of it or not.    (5O)

Not wanting our OHS efforts to fade into oblivion, I asked a subset of the volunteers to participate in a community experiment, which — at Doug’s suggestion — we called the OHS Launch Community. Our goal was not to develop the OHS. Our goal was to figure out what we all thought the OHS was. We would devote six-months towards this goal, and then decide what to do afterwards. My theory was that collectively creating an explicit ontology would be a tipping point in the collaborative process. Once we had an ontology, progress on the OHS would flow naturally.    (5P)

My first recruits were Jack and Howard, and Howard agreed to be our ontology master. We had a real, live Ontologist as our ontology master! How could we fail?!    (5Q)

Mixed Results    (5R)

Howard suggested using Protege as our tool for developing an ontology. He argued that the group would find the rigor of a formally expressed ontology useful, and that we could subsequently use the ontology for developing more-intelligent search mechanisms into our knowledge repository.    (5S)

We agreed. Howard and I then created a highly iterative process for developing the formal ontology. Howard would read papers and follow mailing list discussions carefully, construct the ontology, and post updated versions early and often. He would also use Protege on an overhead projector during face-to-face discussions, so that people could watch the ontology evolve in real-time.    (5T)

Howard made enough progress to make things interesting. He developed some preliminary ontologies from some papers he had read, and he demonstrated and explained this work at one of our first meetings. Unfortunately, things suddenly got very busy for him, and he had to drop out of the group.    (5U)

That was the end of the formal ontology part of the experiment, but not of the experiment itself. First, we helped ourselves by collectively agreeing that developing a shared ontology was a worthwhile goal. This, and picking an end-date, helped us eliminate some of the urgency and anxiety about “making progress.” Developing Shared Language can be a frustrating experience, and it was extremely valuable to have group buy-in about its importance up-front.    (5V)

Second, we experimented with a facilitation technique called Dialogue Mapping. Despite my complete lack of experience with this technique (I had literally learned it from Jeff Conklin, its creator, the day before our first meeting), it turned out to be extremely useful. We organized a meeting called, “Ask Doug Anything,” which I facilitated and captured using a tool called Quest Map. It was essentially the Socratic Method in reverse. We asked questions, and Doug answered them. The only way we were allowed to challenge him or make points of our own was in the form of a question.    (5W)

That meeting was a watershed for me, because I finally understood Doug’s definition of a Dynamic Knowledge Repository. (See the dialog map of that discussion.) One of the biggest mistakes people make when discussing Doug’s work is conflating Open Hyperdocument System with Dynamic Knowledge Repository. Most of us had made that mistake, which prevented us from communicating clearly with Doug, and from making progress overall.    (5X)

Epilogue    (5Y)

We ended the Launch Community on November 9, 2001, about five months after it launched. We never completed the ontology experiment to my satisfaction, but I definitely learned many things. We also accomplished many of our other goals. We wanted to be a bootstrapping community, Eating Our Own Dogfood, running a lot of experiments, and keeping records of our experiences. We also wanted to facilitate collaboration between our members, most of whom were tool developers. Among our many accomplishments were:    (5Z)

The experiment was successful enough for me to propose a refined version of the group as an official entity of the Bootstrap Alliance, called the OHS Working Group. The proposal was accepted, but sadly, got sidetracked. (Yet another story for another time.) In many ways, the Blue Oxen collaboratories are the successors to the OHS Working Group experiment. We’ve adopted many of the same principles, especially the importance of Shared Language and bootstrapping.    (64)

I believe, more than ever, that developing shared ontology needs to be an explicit activity when collaborating in any domain. I’ll discuss where or whether Semantic Web technologies fit in, in a later post.    (65)