Transclusions, Path-Based Addressing, and Version Control

Posted on Published by Eugene Eric Kim

The PurpleWiki community has been rumbling recently, thanks largely to the contributions of two members of the Canonical Hackers, Jason Cook and Matthew O’Connor. Jason wrote perplog, an IRC logger that supports Purple Numbers and Transclusions. Matthew hacked a PurpleWiki node manager, then started adding and fixing other stuff, including an XML-RPC interface. Additionally, John Sechrest developed an experimental email interface to PurpleWiki. Lots of great stuff. It’s forcing us to get off our butts and make some long-promised changes and explain some long-undocumented things. Open Source is a wonderful thing.    (117)

A lot of the excitement is because of PurpleWiki‘s support for Transclusions. We had Transclusions in mind when we architected PurpleWiki, but we (or I, at least) didn’t think Transclusions would actually be implemented until much later. However, exactly one year ago today, Chris Dent got the itch and started playing. A few months later, Chris committed some code, and suddenly, we had Transclusions. It was a total hack, but it worked, and it was unexpectedly cool.    (118)

It’s still a hack, and it needs to be cleaned up, but it’s suddenly become a higher priority. First, we have a pretty good idea of how to support Transclusions “correctly.” Second, having had the chance to use Transclusions regularly, we are starting to recognize their utility and want to take greater advantage of that. (See, for example, my early specs for Abelard.) Third, people are starting to get excited about them.    (119)

Transcluding Multiple Chunks    (11A)

Currently, we support Transclusions of individual nodes (paragraphs, headers, list items, etc.) via the following syntax:    (11B)

  [t nid]    (11C)

where nid is the ID of the node you want to transclude. This works fine when you want to transclude small chunks, but at times, it’s useful to be able to transclude multiple chunks on a page. Rather than specify a transclusion for each individual node, it would be nice to have a syntax for specifying a collection of nodes in a single transclusion.    (11D)

Chris proposed the following syntax:    (11E)

  [t nid,nid,nid,...]    (11F)

This is problematic. The current implementation suggests that the transclusion command be replaced by the content identified by the specified NID. This proposal suggests that the command be replaced by both the content and the structure of the content. If you had a document like:    (11G)

  = Plan for World Domination {nid 1} =    (11H)
  # Finish PurpleWiki. {nid 2}    (11I)

and you tried to transclude this content with:    (11J)

  * [t 1, 2]    (11K)

what is the parser supposed to translate this to?    (11L)

A proper solution must be treated as its own structural element within a PurpleWiki document. More importantly, the syntax should capture document-specific context. This contrasts with the current syntax, which ignores document context entirely.    (11M)

Why is document context important? Suppose you have the following task list:    (11N)

  = To Do {nid 3} =    (11O)
  * Buy milk. {nid 4}   * Feed iguana. {nid 5}   * Implement distributed Transclusions. {nid 6}   * Vote in primaries. {nid 7}   * Expose API to Backlinks. {nid 8}    (11P)

Suppose you want to start a PurpleWiki-specific task list, transcluding all of the list items relevant to PurpleWiki (in this case, nodes 6 and 8). The resulting document might look like:    (11Q)

  {title PurpleWikiToDo}    (11R)
  = PurpleWiki To Do {nid 9} =    (11S)
  * [t 6] {nid A}   * [t 8] {nid B}    (11T)

Now, suppose you want to replicate this task list on another page. You could transclude all of the items individually, just as you do on the PurpleWiki To Do page. In this case, a slight variation of Chris’s proposed syntax (a standalone structural element) would simplify that process. (It also raises an interesting question: Which NIDs do you use for the transclusions: 6 and 8, or A and B? Does it make a difference?)    (11U)

However, what I really want to do is say, “Transclude all of the list items on the ‘PurpleWiki To Do’ page.” For this, you want something like XPointer:    (11V)

  [transclude PurpleWikiToDo#xpointer(id("9")/li)]    (11W)

A few observations: First, the command should be on a line by itself, and it should be interpreted as an independent structural element that will be replaced by a set of structural elements and content. I used “transclude” instead of “t” to make the point that these are two different commands. Second, the Transclusion command specifies a range of nodes within a document, as opposed to a document-independent list of nodes. Third, this combines a path-based address with an ID-based address.    (11X)

Fourth (and this is an implementation detail), if we want to support such syntax, it would behoove us to use an XML data model rather than the home grown model we’re currently using. This way, we could easily plug in existing XPointer implementations to do the queries.    (11Y)

Version Control    (11Z)

The fact that Wiki pages are dynamic throws a kink into all of this. The syntax I propose above takes this into account for the most part. Barring major changes to the PurpleWiki To Do page, the transcluded content will include all of the PurpleWiki tasks, even if more items are added later. If you had to list a set of NIDs, then you would have to be diligent about updating that list manually every time the To Do page changed.    (120)

In addition to supporting path-based addressing, we also need to allow people to specify versions in the address. In other words, you may want to transclude a specific version of a node or a set of nodes from a specific version of a document.    (121)

This shouldn’t be too difficult, but there are some complications. The biggie is whether to transclude a node if the node no longer exists in any document. My instinct right now is telling me that yes, it should, but it should make it clear somehow that it’s an orphan node. (See my previous entry on link integrity for more on this.)    (122)

Dialog Mapping Use Cases

Posted on Published by Eugene Eric Kim

The Compendium Dialogue Mapping tool came up in conversation on the Collaboration Collaboratory. In the course of the discussion, I explained that I use Compendium for three purposes:    (XW)

  • Personal note-taking.    (XX)
  • Taking shared notes during meetings.    (XY)
  • Facilitating face-to-face group meetings.    (XZ)

John Sechrest asked in response:    (Y0)

I have a client come in. We talk about what? And that causes what to show up on the compendium screen?    (Y1)

Specifically, what is the social process that compendium is facilitating?    (Y2)

And how is that different than just writing an outline in an editor?    (Y3)

I mean different things by taking notes on a Shared Display versus facilitating group meetings, so I’ll treat them separately in answering John’s questions.    (Y4)

There are three advantages to taking shared notes real-time. First, you know that a record is being kept of the meeting, which is reassuring. Second, you know that the notes are immediately available. Third, you have the chance to validate the notes as they are being taken.    (Y5)

Taking notes on a Shared Display is a great pattern, regardless of the tool you use. In fact, using a Wiki or a program like SubEthaEdit instead of Compendium may be more valuable in some ways because the note-taking can be collective. When I use Compendium to do this, only I can edit the notes.    (Y6)

The advantage that Compendium has over these other tools is the graphical IBIS grammar. IBIS (which can be expressed as a text outline) is slightly more semantically rigorous and slightly more compact than an outline. Graphical IBIS is superior to a textual outline, because the second dimension serves an additional reference for finding what you’re looking for.    (Y7)

Another advantage of the IBIS grammar is that it encourages, even forces participants to consider the underlying questions (a precursor to the Left-Hand Move pattern). This really comes through in facilitated meetings (with facilitators who are trained in Dialogue Mapping).    (Y8)

The most important thing Dialogue Mapping does in a facilitation context is depoliticize discussion. Dialogue Mapping focuses attention on ideas rather than on people by making sure that all ideas are captured and captured anonymously.    (Y9)

Jeff Conklin, the man responsible for graphical IBIS and Dialogue Mapping, has written a book on Dialogue Mapping that does a great job of explaining how it’s best used as well as the underlying theory. I’m not sure if the book is available yet, but I’ll check. In the meantime, I highly recommend Jeff’s Dialog Mapping workshops. I wrote an article a few years back describing one of these workshops.    (YA)

Are Lurkers Bad?

Posted on Published by Eugene Eric Kim

At the OCSI meeting yesterday, David Hartzband asked Johannes Ernst how many people he expected to have on the online workspace. Johannes said that we should keep it small for now, adding, “Lurkers don’t help us.”    (X5)

I’m probably being a bit unfair to Johannes in mentioning it here, but it got me thinking:    (X6)

  • Do lurkers help?    (X7)
  • Do lurkers hurt?    (X8)

My answer to both: Sometimes. Lurkers are part of a group’s latent energy; good things happen when that energy is activated. Lurkers are part of the all-important weak-tie network, and it’s important to keep them engaged, even if engagement does not translate to participation. However, having lots of lurkers as a community goes through its nascent “sausage stage” can hurt if it drives lurkers and other potential participants away.    (X9)

Here’s another question: Are lurkers members of a community? This question is left as an exercise to the reader.    (XA)

OCSI Meeting Synopsis

Posted on Published by Eugene Eric Kim

I was in Anaheim yesterday for the Open Collaborative Services Initiative (OCSI, pronounced “oxy”) workshop, which was part of the OMG Technical Meeting. Johannes Ernst, one of the OCSI organizers, invited me to present my manifesto on collaborative tools (which will be published in Dr. Dobb’s Journal and on the Blue Oxen Associates web site).    (WI)

OCSI is an attempt to get collaborative tool vendors to make their tools more interoperable. One of its early goals is to develop a shared architectural blueprint for describing collaborative tools, perhaps initially in the form of a white paper. This has been a refrain of mine for quite some time, and so I was very glad to participate in the group’s second meeting.    (WJ)

As it turned out, there was a tremendous amount of conceptual synergy in the room. I suppose I shouldn’t have been too surprised. At the beginning of my talk, I explained that one of our beliefs (also known as the The Blue Oxen Way) is that Shared Ontology (which results in Shared Language) is a prerequisite to effective collaboration. OMG is a very strong proponent of Model Driven Architecture, which is essentially an instantiation of Shared Ontology. Not surprisingly, there was universal consensus in the room about developing a shared model of collaboration — both on the human-level (e.g. Blue Oxen‘s work with Pattern Languages) and the system-level (the topic of my manifesto).    (WK)

In his introductory remarks, Johannes made several interesting points:    (WL)

  • The word “collaboration” means many different things to different people. This simply underscores the need for a common vocabulary.    (WM)
  • Collaboration seems to be an “it” topic among CEOs and CIOs. However, as often as they mention collaboration and as important as they claim it is, the collaborative tools market has been flat the past few years. At first, this seems to be a contradiction. However, the number of corporate downloads of free IM clients over the past few years indicates that the need for collaboration is real. One of the problems is that tools are not interoperable enough.    (WN)
  • There is no horizontal industry initiative for improving interoperability of collaborative tools. However, several vertical industries have expressed interest. One of the challenges is to get the different industries to realize that they share common needs so as not to duplicate efforts.    (WO)
  • Johannes chatted with a few tool vendors about this problem. Their response: “That sounds great, but I have a product to get out.” The way to get vendors more serious about interoperability is probably bottoms-up — via the user community.    (WP)
  • In this regard, the Open Source community could play an important role. The prequisite for standards is Shared Language and free implementations. We have the latter, but we don’t have the former. If we created Shared Language and if Open Source tool-builders adopted it, we could build a compelling case for standardization. Johannes feels that it is vital to involve both the proprietary and Open Source communities in the OCSI effort.    (WQ)
  • Collaborative interfaces should be as transparent as telephone numbers. When we see a telephone number, we know what to do, regardless of the underlying service provider, protocol (POTS versus VOIP), type of telephone, etc.    (WR)
  • Cut-and-paste is a type of interoperability between collaborative tools. (A poor one, as I and others noted later in the workshop, but also a relatively effective one — a good example of loose-coupling.)    (WS)

Other talks of note:    (WT)

  • David Hartzband, VP of collaboration technology at , provided a four axes view of collaborative tools: synchronous, asynchronous, inline, and contextual. He also observed two trends in the collaborative tools space: business communications convergence (e.g. telephone integrated with email integrated with your documents, etc.) and enterprise application functional convergence.    (WU)
  • Carol Burt, CEO of 2AB, shared her vision for model-driven access management. Not only could such a model have ramifications for those developing secure applications and those selling security software, it could also potentially plug in to an OCSI model for collaborative tools.    (WV)

At the end of the workshop, Joaquin Miller (the other OCSI co-organizer) led a discussion about the next steps within the OMG umbrella. The consensus seemed to be to propose the formation of an OMG SIG, which could potentially evolve into an OMG Task Force. Not being an OMG member myself, the conversation both baffled and fascinated me at the same time. Nevertheless, the folks there seemed to know what they were talking about, which is always an excellent sign.    (WW)

The next meeting will be at the next OMG Technical Meeting in St. Louis next April. We’ll continue to collaborate via an eRoom set up by David and via the OCSI web site. Our action item for now is to share our individual high-level models of collaborative tools in order to identify commonalities and to serve as straw men for additional discussion.    (WX)