Kwiki::Purple, Wiki Deep Thoughts

Posted on Published by Eugene Eric Kim

My ex-partner-in-crime, Chris Dent, has been busy coding and expounding. Last month, he released Kwiki::Purple, a Purple Numbers plugin for Kwiki. You can play with it on his test site.    (IFZ)

This is fantastic news on a number of fronts. First, it’s further validation of our strategy to have the ideas take over the world, not the code. As I’ve said from the beginning, the purpose of PurpleWiki is to be a vehicle for ideas. Our goal was not for PurpleWiki to become the Wiki, but for other Wikis to steal our best ideas. There are now three Wikis with Purple Numbers — Kwiki, Zwiki, and PurpleWiki — with hopefully more to come.    (IG0)

Second, the fact that Chris was able to implement this as a Kwiki plugin makes Kwiki a more viable option for Blue Oxen Associates as its Wiki platform of the future. This is a good thing for many reasons.    (IG1)

Chris has also been doing some expounding on Wikis. His entry, “Why Wiki?”, is old hat for folks who know Chris, but it’s a nice, clean summary of his views for those who don’t. His framing of augmentation versus automation (discussed in much more detail in his paper, “The Computer As Tool: From Interaction to Augmentation”) is powerful, and I’ve borrowed it in my own thinking and writing. I also liked this line:    (IG2)

Architecting these sorts of tools may not solve poverty and hunger, or alleviate suffering in the aftermath of a disaster, but the tools can augment people actively doing that work. I happen to be good at making the tools go, so that’s where I look to fit myself into the puzzle.    (IG3)

Chris’s thoughts on Wikis as an external cache is another good piece. Two quick comments. First, viewing Wikis as an external cache reveals an important constraint. They are most valuable to folks who are already immersed in a conversation, because those folks already have some context that aids them in exploring the Wiki. For example, Wikis used for self-documenting events are not so good at involving those who did not participate, but they are extremely valuable for those who did. At the same time, they are better than nothing. If someone is motivated enough, they can use Wikis as a springboard for acquiring the context they need, and thus gain value that way. Think Out Loud is good.    (IG4)

Second, Chris writes:    (IG5)

I’ve found that in order for outboard processing to work there’s several design and process guidelines that have to be reached. Here are some: interaction must be highly responsive, noise in the interface must be minimized, structural mechanics and metaphors in content need to be consisent, names must have value, it must be there when you want it, when there is a shared brain its context is shared as well (e.g when some members of the company have a discussion about design it it is done in an archivable fashion).    (IG6)

(Emphasis is mine.) It’s ironic that Chris cites highly responsive interaction as a requirement for collaboration on an asynchronous medium to work. I agree that this is an important pattern of effective collaboration, but I wouldn’t go so far as to say it’s a requirement. There is an alternative mode, one that emphasizes deep thinking augmented by infrequent, but deep interactions. A big void in the collaborative space are tools that augment this mode of interaction. See my design notes on Abelard for an example of what such a tool might look like.    (IG7)

Stopover in Bloomington

Posted on Published by Eugene Eric Kim

On my way from Fort Wright to St. Louis, I stopped over in Bloomington to have lunch with Chris Dent and some of his colleagues, Joe Blaylock, Kevin Bohan, and Matthew O’Connor. Matthew is one of The Canonical Hackers.    (1YQ)

Lunch conversation was good — spent two hours longer than I had planned. I especially enjoyed meeting Matthew, as well as Paul Visscher and Jason Cook a few nights earlier. You can gather a surprising amount from interacting with folks via email alone, but it’s still only a partial picture. It was good to finally meet these guys in person, and to get a sense of their personalities and passions.    (1YR)

Arrived at Scott Foehner‘s place in St. Louis at around 8:30pm. Had dinner on The Hill at an Italian restaurant called Via’s, then went to Milo’s, a neighborhood bar, for drinks. I was surprised to learn that folks in St. Louis brew beers other than Budweiser. Had a Schlafly’s there, which was very good.    (1YS)

The Future of Purple: Distributed Transclusions

Posted on Published by Eugene Eric Kim

One of my main interests is collaborative tool interoperability. In that vein, I’ve participated in many conversations, but I’ve been actively involved in two initiatives: Purple Numbers and Identity Commons. On the surface, these two technologies solve different problems, but when you dig a bit deeper, you can actually combine the two to do some very interesting things.    (1N1)

Purple Numbers enable granular addressability, which enables you to attach metadata to finer-grained chunks of data. Instead of having an author of a document, you have authors for each paragraph of a document. This is an especially useful concept for collaborative authoring tools, such as Wikis (hence PurpleWiki).    (1N2)

If you can address granular chunks of data, you can also transclude them. This is generally more useful than transcluding an entire document. (You are more likely to want to quote a sentence from a paper than the entire paper.) So, one consequence of tools that support Purple Numbers is that they also support granular transclusions.    (1N3)

One of the limitations of Purple Numbers as currently implemented is that addresses are only unique in a local context. What we really want are globally unique, persistent Purple Numbers. That would allow me to transclude a paragraph from another blog into this one. If that paragraph moves, the transclusion should still work.    (1N4)

Guess what. XRIs are globally unique, persistent addresses. In fact, they were specifically designed to address data. (See the excellent whitepaper, “The Dataweb: An Introduction to XDI” for more on this.) In other words, we can use XRIs for Purple Numbers.    (1N5)

Even better, Identity Commons is developing schemas for link contracts that allow you to attach permissions with your data. While Identity Commons will use these link contracts for protecting profile data, they could certainly be used for other things as well, such as protecting purpled data.    (1N6)

Suppose Alice writes a paper, and she only wants Bob and Carl to see it. She could attach permissions to the entire paper that says as much. Now Bob wants to write a paper, and he wants to quote a paragraph from Alice’s paper via a transclusion. However, he also wants to make his paper available to the public. So, he transcludes Alice’s paragraph and makes his paper publically available. When most people view the paper, they’ll see everything except for Alice’s paragraph because the link contract associated with that paragraph denies them permission. However, when Bob or Carl view the paper, they see everything, including Alice’s paragraph, because they have permission to view that.    (1N7)

More thinking needs to be done. This originally started as a fancy, but a conversation with Gabe Wachob over IRC a few months helped convince me that using XRIs for Purple Numbers was not only possible, but a good application. Two issues are:    (1N8)

  • What should the Purple Numbers look like? It’s the age-old question. XRI e-numbers are definitely not pleasing to the eye.    (1N9)
  • What should the numbers do when you click on them? The way it works right now is that the addresses are named anchors. But this won’t work out-of-the-box with XRIs, because most XRIs are not valid XML IDs.    (1NA)

Chris Dent has mentioned handles on several occasions as another direction we could go. We need to examine this possibility as well.    (1NB)

Tim Bray on Purple Numbers

Posted on Published by Eugene Eric Kim

I’ve long been a fan of Tim Bray‘s work, and although we had never crossed paths before, I had always assumed it would be instigated by me making some comment about something he had done or written about. (A few months back, I emailed him about ballparks in the Bay Area, because we seem to share a love for the sport, but that doesn’t count.) So I was surprised and pleased to see that Tim had made first contact and had (temporarily, as it turned out) implemented Purple Numbers on his blog. Not surprisingly, this is starting to generate some talk in the blogosphere. In particular, see:    (1G9)

Chris has done an excellent job of quickly addressing many of these issues. I’ll just toss in a few thoughts and references here and will look forward to more feedback.    (1GE)

Stable, Immutable IDs    (1GF)

Several people have pointed out that the IDs need to be stable. In other words, as the paragraphs move around or as new ones are added or deleted, the IDs stick to their original paragraphs. This was a fundamental motivation for Engelbart’s statement identifiers (SIDs), which we have renamed node identifiers (NID).    (1GG)

Both Purple (which needs updating) and PurpleWiki handle this correctly. You’ll notice that the addresses are stable on my blog and on Chris’s, as well as on all of the PurpleWiki installations (e.g. Collab:HomePage, PurpleWiki:HomePage). It’s because we use plugins for our respective blog software that call PurpleWiki‘s parser, which manages NIDs for us.    (1GH)

By using PurpleWiki to handle the purpling, blog content has NIDs unique to both the blog and the Wiki, which allows us to do fun stuff like transclude between the two. We get a similar effect with perplog, the excellent IRC logger written by Paul Visscher and other members of The Canonical Hackers. For example, check out Planetwork:Main Page and the chat logs over there.    (1GI)

Mark Nottingham noted that even if the NIDs are stable in the sense that they are attached to certain paragraphs, the paragraphs themselves are not semantically stable. This is a point on which I’ve ruminated in the past, and we still don’t have a good solution to it.    (1GJ)

History and Other Worthy Projects    (1GK)

I’ve written up my own brief history of Purple Numbers, which fills in some holes in Chris’s account. In it, I mention Murray Altheim‘s plink. Plink is no longer available on the net, because it has been subsumed into his latest project, Ceryle. Everything that I’ve seen about Ceryle so far kicks butt, so if you’d like to see it too, please drop Murray an email and encourage him to hurry up and finish his thesis so that he can make Ceryle available! You can tell him I sent you, and you can forward him the link to this paragraph; he’ll understand.    (1GL)

Matt Schneider is the creator of the PurpleSlurple purple numbering proxy, which will add Purple Numbers to any (well, most) documents on the Web. PurpleSlurple deserves a lot of credit for spreading the meme.    (1GM)

Mike Mell implemented Purple Numbers in ZWiki for last year’s Planetwork Conference (see ZWiki:ZwikiAndPurpleNumbers), which in turn influenced the evolution of Purple Numbers in PurpleWiki. Mike and Matt have both experimented with JavaScript for making the numbers less intrusive.    (1GN)

The latest version of the Compendium Dialogue Mapping tool exports HTML maps with Purple Numbers.    (1GO)

In addition to Murray, Matt, and Mike, several other members of the Blue Oxen Associates Collaboration Collaboratory have contributed to the evolution of Purple Numbers, especially Peter Jones, Jack Park, and Bill Seitz. In addition to his contributions to the technical and philosophical discussions, Peter wrote the hilarious Hymn of the Church Of Purple and excerpts of the Book of the Church Of Purple.    (1GP)

Finally, many good folks in the blogosphere have helped spread the meme in many subtle ways, particularly those noted connectors Seb Paquet and Clay Shirky.    (1GQ)

Evangelism and The Big Picture    (1GR)

Okay, that last section was starting to sound like an award acceptance speech, and although none of us have won any awards, one thing is clear: The contributions of many have vastly improved this simple, but valuable tool. I’m hoping that momentum picks up even more with these recent perturbances. I’m especially heartened to see experiments for improving the look-and-feel.    (1GS)

I want to quibble with one thing that Chris Dent said in his most recent account. (When we have distributed Purple Numbers, I’ll be able to transclude it, but for now, you’ll just have to live with the cut-and-paste):    (1GT)

When he [Eugene] and I got together to do PurpleWiki, we were primarily shooting for granular addressability. Once we got that working, I started getting all jazzed about somehow, maybe, someday, being able to do Transclusion. Eugene was into the idea but I felt somehow that he didn’t quite get it. Since then we’ve implemented Transclusion and new people have come along with ideas of things to do that I’m sure I don’t quite get, but are probably a next step that will be great.    (1GU)

There are many things I don’t quite get, but Transclusions are not one of them, at least at the level we first implemented them. That’s okay, though. Ted Nelson felt the same way about my understanding of Transclusions as Chris did, although for different reasons. (And, I suspect that Ted would have had the same opinions of Chris.) I mention this here not so much because I want to correct the record, but because it gives me an excuse to tell some anecdotes and to reveal a bit about myself.    (1GV)

Anyone who knows Doug Engelbart knows that he complains a lot. The beauty of being an acknowledged pioneer and visionary is that people pay attention, even if they they don’t think much of those complaints. When I first began working with Doug in early 2000, I would occasionally write up small papers and put them on the Web. Doug would complain that they didn’t have Purple Numbers. At the time, I recognized the value of granular addresses, but didn’t think they were worth the trouble to add them to my documents. I also didn’t think they were the “right” solution. Nevertheless, because Doug was Doug, I decided to throw him a bone. So I spent a few hours writing Purple (most of which was spent learning XSLT), and started posting documents with Purple Numbers.    (1GW)

Then, a funny thing happened. I got used to them. I got so used to them, I wanted them everywhere.    (1GX)

A few people just get Purple Numbers right away. Murray was probably the first of those not originally in the Engelbart crowd to do so; Chris followed soon thereafter, as did Matt. The vast majority of folks get the concept, but don’t really find them important until they start using them. Then, like me, they want them everywhere. Getting people past that first step is crucial.    (1GY)

A few nights ago, I had a late night conversation with Gabe Wachob (chair of the OASIS XRI committee) on IRC. (This eventually led to a conversation between Chris and me, which led to Chris’s blog entry, which led to Tim discovering Purple Numbers, which led to this entry. Think Out Loud is an amazing thing.) Gabe knew what Purple Numbers were, but hadn’t thought twice about them. I had wanted to ask him some questions about using XRI addresses as identifiers, and in order to do so, I gave him a quick demonstration of Transclusions. The light bulb went off; all of a sudden, he really, truly got it.    (1GZ)

Richard Gabriel, one of our advisors, is well known for his Worse Is Better essays (among other things). I think Purple Numbers are an outstanding example of Worse Is Better. They fulfill an immediate need, and they cause us to think more deeply about some of the underlying issues. I’d like to see Purple Numbers all over the place, but I’d also like to see a group of deep thinkers and tinkerers consider and evolve the concept. It’s part of a larger philosophy that I like to call The Blue Oxen Way.    (1H0)

This last point is extremely important. Chris has thankfully been a much more enthusiastic evangelist of Purple Numbers than I have, and in the past he’s called me “ambivalent” about Purple Numbers. That’s not so far from the truth. It’s not that I’m any less enthusiastic about Purple Numbers themselves — I am a card-carrying member of the Church Of Purple, and the current attention and potential for wider usage thrill me. However, I’m cautious about evangelizing Purple Numbers, because I don’t want people to get too caught up in the tool itself and forget about the bigger picture. It’s the reason I didn’t mention Purple Numbers at all in my manifesto.    (1H1)

At the Planetwork forum two weeks ago, Fen Labalme, Victor Grey, and I gave the first public demo of a working Identity Commons Single Sign-On system. We were tickled pink by the demo, which to everyone else looked just like any other login system. The reason we were so excited was that we knew the system used an underlying infrastructure that would eventually enable much greater things. The demo itself, unfortunately, didn’t convey that to anyone who didn’t already understand this.    (1H2)

I’m probably a bit oversensitive about this sort of thing, and I’m constantly seeking better balance. But it’s always in the back of my mind. When I talk to people about Blue Oxen Associates, I usually spend more time talking about the sociological aspect of collaboration rather than the tools, even though I have plenty to say about the latter. Can Purple Numbers make the world a better place? I truly, honestly, believe that they can. (This is a topic for another day.) But when I see groups that excel in collaboration (or conversely, those that stink at it), Purple Numbers are usually the furthest thing from my mind. Much more important is the need to identify and understand these patterns of collaboration (of which tool usage is an important part).    (1H3)

Transclusions, Path-Based Addressing, and Version Control

Posted on Published by Eugene Eric Kim

The PurpleWiki community has been rumbling recently, thanks largely to the contributions of two members of the Canonical Hackers, Jason Cook and Matthew O’Connor. Jason wrote perplog, an IRC logger that supports Purple Numbers and Transclusions. Matthew hacked a PurpleWiki node manager, then started adding and fixing other stuff, including an XML-RPC interface. Additionally, John Sechrest developed an experimental email interface to PurpleWiki. Lots of great stuff. It’s forcing us to get off our butts and make some long-promised changes and explain some long-undocumented things. Open Source is a wonderful thing.    (117)

A lot of the excitement is because of PurpleWiki‘s support for Transclusions. We had Transclusions in mind when we architected PurpleWiki, but we (or I, at least) didn’t think Transclusions would actually be implemented until much later. However, exactly one year ago today, Chris Dent got the itch and started playing. A few months later, Chris committed some code, and suddenly, we had Transclusions. It was a total hack, but it worked, and it was unexpectedly cool.    (118)

It’s still a hack, and it needs to be cleaned up, but it’s suddenly become a higher priority. First, we have a pretty good idea of how to support Transclusions “correctly.” Second, having had the chance to use Transclusions regularly, we are starting to recognize their utility and want to take greater advantage of that. (See, for example, my early specs for Abelard.) Third, people are starting to get excited about them.    (119)

Transcluding Multiple Chunks    (11A)

Currently, we support Transclusions of individual nodes (paragraphs, headers, list items, etc.) via the following syntax:    (11B)

  [t nid]    (11C)

where nid is the ID of the node you want to transclude. This works fine when you want to transclude small chunks, but at times, it’s useful to be able to transclude multiple chunks on a page. Rather than specify a transclusion for each individual node, it would be nice to have a syntax for specifying a collection of nodes in a single transclusion.    (11D)

Chris proposed the following syntax:    (11E)

  [t nid,nid,nid,...]    (11F)

This is problematic. The current implementation suggests that the transclusion command be replaced by the content identified by the specified NID. This proposal suggests that the command be replaced by both the content and the structure of the content. If you had a document like:    (11G)

  = Plan for World Domination {nid 1} =    (11H)
  # Finish PurpleWiki. {nid 2}    (11I)

and you tried to transclude this content with:    (11J)

  * [t 1, 2]    (11K)

what is the parser supposed to translate this to?    (11L)

A proper solution must be treated as its own structural element within a PurpleWiki document. More importantly, the syntax should capture document-specific context. This contrasts with the current syntax, which ignores document context entirely.    (11M)

Why is document context important? Suppose you have the following task list:    (11N)

  = To Do {nid 3} =    (11O)
  * Buy milk. {nid 4}   * Feed iguana. {nid 5}   * Implement distributed Transclusions. {nid 6}   * Vote in primaries. {nid 7}   * Expose API to Backlinks. {nid 8}    (11P)

Suppose you want to start a PurpleWiki-specific task list, transcluding all of the list items relevant to PurpleWiki (in this case, nodes 6 and 8). The resulting document might look like:    (11Q)

  {title PurpleWikiToDo}    (11R)
  = PurpleWiki To Do {nid 9} =    (11S)
  * [t 6] {nid A}   * [t 8] {nid B}    (11T)

Now, suppose you want to replicate this task list on another page. You could transclude all of the items individually, just as you do on the PurpleWiki To Do page. In this case, a slight variation of Chris’s proposed syntax (a standalone structural element) would simplify that process. (It also raises an interesting question: Which NIDs do you use for the transclusions: 6 and 8, or A and B? Does it make a difference?)    (11U)

However, what I really want to do is say, “Transclude all of the list items on the ‘PurpleWiki To Do’ page.” For this, you want something like XPointer:    (11V)

  [transclude PurpleWikiToDo#xpointer(id("9")/li)]    (11W)

A few observations: First, the command should be on a line by itself, and it should be interpreted as an independent structural element that will be replaced by a set of structural elements and content. I used “transclude” instead of “t” to make the point that these are two different commands. Second, the Transclusion command specifies a range of nodes within a document, as opposed to a document-independent list of nodes. Third, this combines a path-based address with an ID-based address.    (11X)

Fourth (and this is an implementation detail), if we want to support such syntax, it would behoove us to use an XML data model rather than the home grown model we’re currently using. This way, we could easily plug in existing XPointer implementations to do the queries.    (11Y)

Version Control    (11Z)

The fact that Wiki pages are dynamic throws a kink into all of this. The syntax I propose above takes this into account for the most part. Barring major changes to the PurpleWiki To Do page, the transcluded content will include all of the PurpleWiki tasks, even if more items are added later. If you had to list a set of NIDs, then you would have to be diligent about updating that list manually every time the To Do page changed.    (120)

In addition to supporting path-based addressing, we also need to allow people to specify versions in the address. In other words, you may want to transclude a specific version of a node or a set of nodes from a specific version of a document.    (121)

This shouldn’t be too difficult, but there are some complications. The biggie is whether to transclude a node if the node no longer exists in any document. My instinct right now is telling me that yes, it should, but it should make it clear somehow that it’s an orphan node. (See my previous entry on link integrity for more on this.)    (122)