Purple Numbers and WIKIWYG

For a while, it was looking like I was going to break another personal blogging record last month, then things got so busy I had zero time to blog whatsoever. That means I’m in catch up mode again, so as usual, I’ll post in reverse chronological order (which in the blogosphere is really reverse reverse chronological order).    (JYN)

Yesterday, I spent the afternoon at Socialtext, where they were having an all-hands meeting. They graciously invited me to participate in the Open Space segment of their gathering, which meant quality time with Chris Dent and a rare opportunity to evangelize the Church Of Purple together. Of course, Chris has been spreading the Purple religion at Socialtext for a while now, so it wasn’t as much about evangelism as it was about next steps.    (JYO)

As I’ve mentioned many times before, browser-based WYSIWYG editors are an exciting development because they allow us to make Purple Numbers transparent in the authoring process. Right now, when you edit a PurpleWiki page, you see the node ID tags (e.g. {nid 123}). This is impossible to get around with the default browser text-editing widget. However, with a WYSIWYG editor, you can hide the Purple Numbers while still maintaining their associations with a node behind the scenes.    (JYP)

That’s the theory, anyway. In particular, I’ve been excited about WIKIWYG ever since Ross Mayfield showed me an early prototype last August. I had a personal bias, since Chris Dent and Matt Liggett helped write it, as did Casey West and the inimitable Brian Ingerson, whom I finally met last weekend at Tag Camp.    (JYQ)

Yesterday afternoon, we discussed Purple Numbers and WIKIWYG, and it was good. Then in the evening, Ingy and I spent a few hours trying to get WIKIWYG integrated into PurpleWiki.    (JYR)

We didn’t quite make it. Our biggest roadblock was a bug we discovered in Mozilla’s design mode that we can’t do much about. (My days of statically typed languages are well behind me.) But, we got something somewhat working, and I learned a heckuvalot. You can play with our semi-working demo.    (JYS)

WIKIWYG seems well-architected and is easy to customize. For folks with relatively standard Wiki editing requirements, I highly encourage you to play with it. PurpleWiki has some special formatting funkiness (mainly due to the Purple Numbers), but we were able to get around this fairly easily. (This was also true thanks to PurpleWiki‘s model of parsing to an intermediate data structure, then using view drivers to serialize. I wish more Wiki engines did this. I know Magnus Manske is thinking about doing this for Mediawiki, and I think Janne Jalkanen is already doing it with JSPWiki.)    (JYT)

The Mozilla bug annoyed me, because it’s a show-stopper in some ways, and there’s not much I can do about it. I didn’t realize it, but all of the JavaScript WYSIWYG widgets actually switch to the browser’s “design mode” in order to handle WYSIWYG editing. As with many HTML editors, design mode does not handle structure cleanly, and you end up getting weird artifacts such as spurious break tags. Our problem was that we serialize node ID information as id attributes in the HTML tags. However, Firefox does not maintain those attributes correctly when you move content around.    (JYU)

I’ll report the bug (if folks have suggestions as to the best way to bring this to the right people’s attention, let me know), but it also puts the kibosh on my hopes for WIKIWYG and Purple Numbers. Even if the bug is fixed in the next version of Firefox, we’re still prey to all the folks using older versions as well as Internet Explorer or Safari, which have their own problems with design mode.    (JYV)

Chris and I discussed one workaround that I’m still pondering: render the Purple Number and have users be responsible for maintaining the association with the nodes. That’s the status quo, except users are doing it in WikiText rather than in WYSIWYG. Doing it in WYSIWYG certainly lowers the bar, and it’s probably the next best thing for us to do.    (JYW)

Fighting WikiSpam: Eaton and Shared Blacklists

WikiSym 2005 was awesome. Massive props to Dirk Riehle and the program committee for throwing an outstanding event and drawing tons of great, great people. With Wikimania last August and WikiSym this past week, the Wiki community is really starting to gel. And it’s about time. Can you believe Wikis are 10 years old?    (JXD)

Now the bad news: I walked away with some action items. How do I get myself into these messes?!    (JXE)

The first action item can be traced back to an ad hoc meeting that happened at Wikimania regarding WikiSpam. On August 6, a group of Wiki developers — me (PurpleWiki), Alex Schroeder (OddMuse), Brion Vibber (Mediawiki), Thomas Waldmann (MoinMoin), Sven Dowideit (TWiki), Janne Jalkanen (JSPWiki) — along with John Breslin and Jochen Topf, got together to discuss ways we could collaborate on fighting WikiSpam. Our goal was to identify the simplest possible first step and not to get mired in process discussions.    (JXF)

Since all of us were already maintaining URL blacklists, we decided to merge them and host it as a Sourceforge project. We agreed on a standard format (which I’ll document and post soon), and we agreed to send our respective lists to Alex, who already has scripts to slice, dice, and merge.    (JXG)

One of my action items then was to create the Sourceforge project. I did that immediately, but for some reason, the project was rejected. Thus began a month-long go-around with Sourceforge support where I tried to discover why they had rejected the proposal. In the end, the project was approved, and I never got an answer as to why it was rejected in the first place. At that point, I was mired in other work, and so I never followed up.    (JXH)

WikiSym was the kick in the butt I needed to follow-up. On Sunday, Sunir Shah hosted an antispam workshop, which about 40 people attended. First, Sunir reviewed techniques (many of which are listed at MeatBall:WikiSpam). Then we broke out.    (JXI)

In my breakout, I described what we had agreed on at Wikimania. Then Peter Kaminski described a very cute idea he had for making it easy to fight WikiSpam. In a nutshell, Peter suggested we write a simple drop-in replacement CGI wrapper that would filter a POST payload for spam and call the real CGI script — be it a Wiki, a blog, or anything else — if the payload were spam-free. Such a wrapper would enable users to install spam-protection for any CGI script without having to write a single line of code and without having to do any complex configuration. It wouldn’t require any special access to your web server, since it would just be a CGI script. And you could easily add other spam-fighting measures, such as throttling and IP blacklists.    (JXJ)

I thought it was a brilliant idea. So Peter and I sat down afterwards and whipped it up. Took about an hour. It’s called Eaton, it works, and it’s Public Domain. Peter Kaminski has already blogged about it, and there’s some important commentary there from Jay Allen, the creator of MT-Blacklist.    (JXK)

It’s a proof of concept, and it won’t scale. It can and should be improved, and I’d encourage folks to do so. Nevertheless, it’s pretty cool. Bravo to Peter for a very clever idea.    (JXL)

By the way, the first person to figure out the origins of the name “Eaton” wins a cookie.    (JXM)

Patterns at WikiMania 2005

When I first met Christine Peterson (now a Blue Oxen Associates advisor), she told me the story about coining “Open Source.” The sign of a good name, she explained, is when people naturally start using it on their own. At a meeting of developers and evangelists in early 1998, rather than argue strongly in favor of the term, she introduced it subtly. Although the response wasn’t enthusiastic at the first, everyone in the room found themselves using the term, and by the end of the meeting, they all agreed to evangelize it.    (JMS)

Similarly, my strategy for introducing and identifying patterns of high-performance collaboration is to subversively introduce patterns into various communities and then to listen. If people naturally use a pattern in conversation, the name is probably good and the pattern itself is probably real and repeating. As people become familiar with the concept, they are more likely to identify and name other patterns. Over time, the language shifts your thinking, giving you a cognitive framework for thinking about, talking about, and improving collaboration and collaborative tools. Moreover, the process itself is iterative and collaborative, which is both the right way to develop Pattern Languages and also another application of collaborative patterns.    (JMT)

I’ve been giving some variation of a stock talk on patterns for over a year now, including last week at Wikimania 2005. It usually consists of a quick introduction, a few examples, and an interactive portion where I tease out patterns from the audience. The audience banter is always the best part. It’s always different, and it’s provided me with entertaining anecdotes, new patterns, and better pattern names.    (JMU)

Last week, I mentioned four patterns that Wikis facilitate: Permission To Participate, Shared Display, Visible Pulse, and Working Draft. Tim Starling followed my talk with an overview of Mediawiki development, and when he mentioned their IRC channel, he said, “This is our community’s Visible Pulse.” I love it when the process works!    (JMV)

The discussion teased out other patterns, especially Celebration and Initiation. (Linda Rising and Mary Lynn Manns mentions both of these in their book. They have a better name for the latter, but I don’t remember it off-hand.) One person told a great story about both. His team met for the first time in Australia, and before embarking on their project, they brewed beer that they planned on drinking after they finished their project.    (JMW)

Other patterns observed and not observed at the conference and within the community:    (JMX)

  • The Celebration at the end of the conference was great, but the Initiation wasn’t particularly remarkable. This isn’t unusual for conferences, where Initiation tends to be in the form of a keynote.    (JMY)
  • One pattern closely related to Initiation is Introductions. At conferences, this generally comes in the form of name badges, which we had at Wikimania. I think there’s a huge opportunity for further facilitating this pattern at conferences, which is something Blue Oxen Associates is working on. Hacking Days also would have been significantly more effective if we simply went around the room each day and quickly introduced ourselves. It’s one of those patterns that sound obvious when you hear about it, but is often forgotten when actually designing an event.    (JMZ)
  • Conferences usually do Water Cooler well, and Wikimania was no exception. We had long lunches, a party on the last night, an IRC channel, and some organized activities in Frankfurt am Main for stragglers following the conference. In particular, the organizers did two things really well. The Haus der Jugend was an excellent choice of venue, because it was an intimate space where social interaction was practically unavoidable. Most of the participants stayed at the hostel, which meant that there was always an interesting conversation to be had by simply going downstairs and hanging out in one of the common rooms. Plus, most of us who stayed there also had roommates, which is not typical for the conferences I attend. The restaurants and bars — touristy though they were — were in very close proximity, which made it easy to grab a beer and talk. (Food is a closely related pattern.) Finally, Hacking Days served as an unintentional Water Cooler, at least for those of us who were not Mediawiki developers. Coming early is something the organizers should encourage more widely next time.    (JN0)
  • I’m very biased towards highly interactive event design, and it seemed like it would have been especially appropriate for a Wiki conference. That said, the panel format worked better than at most conferences, and I think the Wiki culture of Permission To Participate had a lot to do with that. The audience interacted easily with the presenters at all of the talks I attended, and most of the speakers did a nice job of incorporating feedback into their presentations. One thing they did not do well was actively promote and integrate the conference Wiki as a place to take notes and have additional discussions. Again, this seemed surprising for a Wiki conference — yet another indication that making patterns explicit is a good thing.    (JN1)
  • Another favorite — also found in Linda and Mary Lynn’s book — is Spotlight On Others. This runs rampant throughout the Wikipedia community, which is wonderful. Tim cited me when mentioning Visible Pulse, and by the next morning, Sunir Shah had incorporated Permission To Participate into his talk on conflict resolution. I found this time and again in people’s talks. One reason it occurred so often during the conference is that the organizers used a Wiki to develop the program transparently. People watched other people’s talks develop and incorporated their content even before the conference began.    (JN2)
  • A critical pattern, especially for inter-organizational collaboration, is Neutral Space. Wikipedia would not have been successful if it did not have an open content license (which facilitates Neutral Space) and, to some extent, if it were a for-profit company. At the board panel on the last day, several people brought up the question of online ads. On the one hand, ads have the potential of bringing in a tremendous amount of revenue, which could be put to good use for the community. On the other hand, it breaks the Neutral Space pattern that has served Wikipedia so well. The community was more or less split on this issue. My vote: No ads!    (JN3)

One last pattern that I both observed and missed was Users Talk To Developers, a pattern I first described in, “An Introduction to Open Source Communities.” Previously, I criticized the Mediawiki developers for not practicing it enough. With the whole conference finally behind me, I want to both soften and and strengthen my statement.    (JN4)

Many of the Mediawiki developers came to the project as Wikipedia contributors. Brion Vibber, one of the leaders of the project, probably never would have joined had it not been for the Esperanto Wikipedia, of all things. After having more time to interact and observe the developers, I think that on average, community interaction is more prevalent among the Mediawiki developers than it is with many other projects.    (JN5)

That said, it’s still not nearly what it can and should be. During the sessions on politics and developing countries, several panelists complained that the tools had a way to go to meet their needs, and yet, none of the developers were attending their sessions. Hossein Derakhshan noted that techies are generally not interested in issues outside of their sphere.    (JN6)

Not all the blame falls on developers, however. As great as it would have been to see more developers abandoning the technical sessions in favor of the more social ones, it would have been fantastic to see more Wikipedia contributors attend some of the technical sesssions. Both communities need to learn and respect each other’s language if they truly want to engage collaboratively. Bridges are critical to make this work. Note that this applies not only to Mediawiki, but to all Open Source projects.    (JN7)

WikiMania Hackfest Day 4

Bits and tids:    (JM7)

  • I didn’t plan my Hacking Days schedule very well. I missed most of the first day, when the Mediawiki developers apparently made progress on a new metadata design. Days 2 and 3, from which I based most of my criticism, focused on servers and reliability, an area to which I really couldn’t contribute, not because I’m ignorant, but because I’m powerless. This morning, they discussed Single Sign-On and usability, two areas that I do know something about. Sadly, I missed these sessions, because I was too busy spouting on and on about how we really can save the world. Owen Davis, Fen Labalme, Kaliya Hamlin, and the rest of the gang will undoubtedly kick my butt when they read this. In my defense, I managed to talk a bit about Identity Commons later in the day. I also plugged the FLOSS Usability Sprint, and met Zeno Gantner, who’s done some usability studies on Mediawiki.    (JM8)
  • I was one of the featured participants for the afternoon “Wiki developers informal discussion,” along with Ward Cunningham, Sven Dowideit, Christophe Ducamp, and Brion Vibber. Domas Mituzas, Wikimedia Foundation‘s head of operations, asked Ward, “Why Camel Case?” I won’t go into the explanation here — I have a long interview with Ward, to be published eventually, that explains this in detail — but you should know that hating Camel Case is a running joke among this community. I laughed along with everyone else, but when Sven mentioned his desire to remove Camel Case from TWiki, I felt compelled to pipe up. I gave a balanced defense, describing Camel Case’s advantages over free links, but also acknowledging the appropriateness of free links in Wikipedia. Then I got a very amusing introduction to Erik Moeller, one of Mediawiki‘s core contributors and the Wikimedia Foundation‘s chief research officer. Erik had a strongly worded response. It got a bit heated, but never overly so, and I closed by saying that we were in violent agreement. We laughed about it over dinner, but then we got serious again. We also talked about Purple Numbers. I’ve explained many times why I may seem like a poor evangelist, but I think Erik was one of the few people who appreciated my perspective. He was clearly not a big fan of Purple Numbers — as it turns out, he was somewhat familiar with my work — but after hearing my explanation, he responded, “Only intelligent people are going to understand what you just said.” Fair enough. Fortunately, regular folks don’t need to get Granular Addressability for Granular Addressability to become ubiquitous.    (JM9)
  • A group of us broke out into a small group to discuss a Wiki Interchange Format, knowing full well that this is an issue that’s been discussed many times before (Wiki:WikiInterchangeFormat, MeatBall:WikiInterchangeFormat). Nevertheless, I think our discussion was not only constructive, it has a high chance of succeeding. See my summary.    (JMA)
  • Magnus Manske, the original creator of Mediawiki, participated in our Wiki Interchange Format discussion. He also mentioned a clever idea: a “shopping cart” where people could aggregate and possibly export Wiki pages they were interested in.    (JMB)
  • Sven Dowideit demonstrated the prototype WYSIWYG editor for TWiki, based on Kupu. He also showed a WikiText editor with real-time preview, which was pretty slick. Also, Ross Mayfield showed me a prototype editor for KWiki in response to my previous post. Very good to see these things.    (JMC)
  • So many people have come to this gathering to learn from others with different experiences. Granted, all of these experiences center around Wikipedia, but I’m still envious. My neverending quest is for folks interested in collaboration to look beyond their own narrow domains for deeper insights.    (JMD)

WikiMania Hackfest Day 3

Today’s tidbits:    (JLI)

  • Ward Cunningham arrived this morning and gamely participated in the afternoon session, despite the long flight. At one point, he asked the Mediawiki developers what the consequences of a wrong architectural decision would be. The answers were revealing. These guys are faced with a tough problem thanks to scaling issues. By necessity, they have to optimize their architecture based on user behavior. However, this severely restricts their flexibility, because if their predictions about user behavior are wrong, it is extremely expensive to rearchitect, reimplement, and reconfigure. Moreover, Mediawiki really can’t develop agilely, at least not where new user features are concerned, for the same reasons. If they were part of a large company with plentiful IT resources (e.g. Google), then it would be a different story, but they’re not. This also makes Extreme Usability somewhat of a pipe dream.    (JLJ)
  • Funny observation from a Mediawiki developer (Mark) on downtime: They’re the only site that profits from downtime. When Wikipedia goes down, folks assume it’s short on resources and tries to donate stuff.    (JLK)
  • At the FLOSS Usability Sprint last February, we framed the following conflict between Open Source developers and usability practitioners: With usability, the mantra is, “You are not your user.” With Open Source, the mantra is Scratch Your Own Itch. Those two philosophies seem contradictory. However, it’s not so simple. Many new features in Open Source projects are user-driven, although the degree to which this happens largely varies. My impression with Mediawiki from watching the developers work and from talking to various members of the community — developers and users — is that the developers are open to feature suggestions, but are not particularly enthusiastic about user-driven design. Developers will implement suggestions, provided they are posted to Bugzilla. That’s a poor way of doing things, in my opinion. I’m picking on Mediawiki, but it’s a very common attitude in the Open Source world — it came up several times at the usability sprint — and really, in the entire software development community. With Mediawiki, I’m particularly disappointed, because they’ve got this huge, wonderful user community. As I said yesterday, the purely technical problems Mediawiki faces are fascinating in and of themselves. However, the opportunity to evolve some really cool user features should be just as fascinating.    (JLL)
  • Then again, perhaps Mediawiki just isn’t the place for this to happen. As I said above, doing Extreme Usability would be especially hard, because the high overhead imposed by the architecture makes it difficult to develop agilely. Maybe the place to experiment is with smaller communities, and Mediawiki can steal features accordingly. That, after all, is one of the goals behind the Blue Oxen Collaboratories — explore cool ideas in real communities and teams, and then propagate them so they are stolen far and wide. The only problem with this is that it’s almost impossible to duplicate the social effects of scale seen on Wikipedia.    (JLM)
  • I mentioned a “MySQL guy” yesterday. His name is Jan Kneschke, and while he works for MySQL, he’s really here to help Mediawiki with performance issues. One way he’s doing that is by advising them on migrating to his high-performance web server, lighttpd (pronounced “lighty”). lighttpd is a relatively new implementation of an old idea: rather than prefork or thread a web server, as Apache does, use a single event loop, which saves you tons of memory and CPU, especially at scale. It’s got some other cool performance tweaks as well. Jan is very sharp, and lighttpd seems to be taking off. Another high-profile site that uses it is Ruby On Rails. One thing I find amusing is that FastCGI, which has been around forever, seems to be making a big comeback. Ruby On Rails, PHP, Sympa, and many other high-profile tools use it. It’s amazing how old things can fly under the radar forever, then suddenly find new life.    (JLN)
  • Over and over again, I see that proficiency in one area doesn’t necessarily translate into others, even if they seem like they should. Folks in the Wikipedia community are incredibly knowledgable about online, emergent communities. However, judging from the conference design, they know very little about facilitating similar emergent patterns at face-to-face gatherings. Again, I’m picking on them, only because I’m here, but this is a widespread problem, and it further validates what Blue Oxen Associates is trying to achieve. Two things that the organizers here have done very, very right: Bring together great people and establish a wonderful culture of openness.    (JLO)