The Unexpected Pleasures of Open, Transparent Processes

Every project has a team. How those teams form is an interesting question. For the most part, the “leader” of a project decides who is on or off a team, and then they go off on their merry way.    (N2B)

If that work is done openly and transparently, then that opens up the possibility of attracting other contributors. This is the essential goal of movement-building, where the desire is to get other people to act. It’s also what makes Open Source software projects and Wikis work.    (N2C)

The question is, how open and transparent should the process be? Being fully open invites distraction. When the Chandler Project was first announced, list activity went through the roof, and the signal-to-noise ratio was very low. People were galvanized by Mitch Kapor‘s involvement and, of course, everyone had an opinion. It was a dangerous time for the project, because the goals were not concrete, and the process for how to reach concreteness had not yet been established.    (N2D)

How do you balance the desire to be open (and attract outside contributors) with the need to get things done? The key is clarity of vision and process, as well as a realistic assessment of the risks and rewards.    (N2E)

When Doug Engelbart asked me to lead the HyperScope project in 2006, I knew that it was imperative that we deliver. (Visionaries have a sometimes deserved reputation for never actually delivering a product.) At the same time, we really wanted to spread the core ideas widely and galvanize the community.    (N2F)

We had already decided that the project would be Open Source, which meant that some of the processes would inherently be open — open source code repository, open mailing list, open Wiki, and so forth. We also instituted good practices — regularly posting summaries to the community, carrying on much of our activity online (a key principle of the Apache community), and so forth.    (N2G)

I also decided that our weekly face-to-face meetings would be open as well. This was the biggest risk for a few reasons. Doug was going to attend the meetings, and there was a possibility that people would come just to meet Doug. This, in my opinion, was actually a good thing, as long as we weren’t overwhelmed with people and as long as guests did not prevent us from getting things done. I reasoned that we wouldn’t be for two reasons. First, Doug isn’t as much of a household name as he should be, certainly not as much as Mitch Kapor for example. (This is sad and the topic of a future post.) Second, neither was I. I have a large social network, but it’s not enormous, and it’s fairly intimate with lots of trust. I didn’t think a huge number of people would see my announcement, and I trusted those who did to do the right thing.    (N2H)

I was confident in my ability to tightly facilitate the meetings, and I also felt strongly that the rewards of openness, in our case, far outweighed the risks. I was also prepared for the possibility that no one would show up, despite the openness.    (N2I)

I was blown away by who did show up. We had a lot of folks from the old Augmentation Research Center, which was an absolute delight. Many people told me afterwards what a delight it was for them to have us pepper them with questions about their 50-year old work. We had many friends and friends-of-friends visit, all of whom greatly added to the conversation.    (N2J)

Before each meeting, I welcomed our guests, but I also explained that it was a working meeting, and I asked that they respect our need to get things done and participate accordingly. For the most part, that’s all the facilitation I needed to do. I occasionally had to reign people in, but I usually found myself actually encouraging our guests to participate.    (N2K)

At our launch party, I gave special T-shirts to each team member. Two of the people on the team and on the T-shirt were people from the community who found out about our work on their and dropped in. One was John Deneen, a long-time fan of Doug’s, who videotaped and photographed all of our meetings. The second was Craig Latta, a Smalltalk guru who had worked on a variety of old-school hypertext projects in the past and who ended up helping us tremendously on a variety of technical challenges.    (N2L)  T    (N2M)

When I work with groups, I often encourage them to challenge their own assumptions about openness and transparency. Extra effort is required, and sometimes, the hoped-for benefits do not occur. However, more often than not, folks are pleasantly surprised — sometimes even amazed — by what emerges.    (N2N)

barx: A Ruby XRI Resolver

Last month, Victor Grey and Kermit Snelson announced barx, the first full implementation of the XRI 2.0 draft specification (working draft 11, for those of you keeping track). I finally downloaded and started playing with it tonight; it’s very nice. Most OpenID implementations are using a proxy hack to support i-names, but as real XRI implementations start to come out, we’ll start seeing many more interesting applications.    (MS5)

I’ve started to port barx over to Perl and will hopefully have it completed by IIW next week. Yes, I’m coding again. I’ve been sitting on a slew of year-old ideas that need to get implemented, and I’m tired of being a preacher instead of a do-er (at least when it comes to code). It’s against my instincts, and I don’t have enough of an audience to leverage the Lazy Web.    (MS6)

Besides, I was starting to miss it. Over the last few years, I’ve built a reputation as someone who knows a bit about collaboration, not just about tools, and that’s been really gratifying. It’s also helped me feel okay about reminding people that I still know a bit about tools as well. Plus, a lot of things have been stoking the fire recently. I was managing the HyperScope project last year and the Grantsfire project this year, both of which are conceptually and technically interesting. I never stopped reading code, and a lot of my friends are developers. What really kicked things into gear for me, though, was stepping in as an emergency developer for Grantsfire and watching Linus Torvalds‘s git talk.    (MS7)

I started playing with a bunch of ideas at once, but I’m focusing on Grantsfire and the Digital Identity stuff now. Stay tuned, and if you want to hack with me over the next few weeks, either face-to-face or remotely, ping me.    (MS8)

MySQL, Open Source, and Trust

When Jonathan Cheyer wasn’t working with me and Brad Neuberg on HyperScope, he was scrapping away at his day job as Solid Information Technology‘s Open Source community manager. Despite having to deflect my endless teasing about revoking his hacker membership card for becoming a “marketing guy,” he’s been an excellent source of stories and insights about the nature of Open Source communities and collaboration. (I’m less concerned about his hacker cred than I am about him being a die-hard Celtics fan. Sad, very sad.)    (MII)

Jonathan recently blogged about some controversy surrounding MySQL AB‘s decision not to distribute source tarballs of its Enterprise Edition. Why is this seemingly minor move such a big deal? He explains:    (MIJ)

It’s about the importance of being earnest in what you do. Being an open source company is about a lot more than just slapping a GPL license on your software and handing it out. It’s about building a relationship with the community that is using, playing, testing, and improving your software. As anyone who is married knows well, this can only be done through ongoing, continual trust and transparency between the two parties. Trust is built by being dependable, and by telling the other person things that sound honest and real. Trust is improved by transparency, which is opening yourself to the other person. Adding an artificial means of inconvenience to the community in obtaining bits does nothing to help customers and only reduces transparency as seen by the community.    (MIK)

I’m amazed at how often good companies with a strong understanding of Open Source forget this. I think it’s indicative of the ongoing tensions that businesses must balance, and it speaks even more favorably of companies that manage to consistently uphold their Open Source values even in the face of these difficult tensions.    (MIL)

I don’t have any first-hand insights into MySQL as an Open Source project. I do know that it’s been a model in the community for doing commercial Open Source for a long time, and I know a bunch of great folks who are involved in that community, Jonathan included. Jonathan sums it up best when he writes:    (MIM)

MySQL AB has been working with the open source community for a long time and a lot of good things have been accomplished as a result of that. There is much to applaud. Along the way, there have been occasional mistakes, and this is one of those times. MySQL risks alienating a community that has been very supportive of them by a misguided move in in their quest to “get more customers”. Make money, make as much as you can, but while you do, don’t forget the lesson of being earnest in your endeavors and staying true to your community.    (MIN)

purple-include: Granular Transclusions for the Common Man

Many thanks to Jonathan Cheyer, Craig Latta, and Kaliya Hamlin for coming to the HyperScope sprint this past weekend, and special thanks to Christina Engelbart for hosting. Also thanks to Thom Cherryhomes and others who hung out with us on IRC. The notes from the day are up on the Wiki, and I put up some pictures as well.    (MEL)    (MEM)

The big news, though, is that Brad, Jonathan, and I wrote a cool hack called purple-include, based on Mark Nottingham‘s most excellent hinclude. It lets you transclude granular chunks of content from any web site by using an img-like tag. Check out the examples. I think this will go a long way in making Transclusions more common on the Web.    (MEN)

You address granular content either by using a fragment identifier that the document author provides (such as a Purple Number) or by using an XPath expression. Thanks to Tony Chang for his cool interactive XPath tester.    (MEO)

The planned next step is to create a Firefox plugin that adds a “Transclude” option when you right click inside of a browser text widget. This will allow you to transclude copied content, rather than paste it. Don’t know whether any of us will get to this soon, so we encourage the lazy web and all you Firefox hackers to beat us to the punch.    (MEP)

This was my first non-trivial foray into JavaScript, and I was disturbed by what I saw. The language itself is not horrible, although its object system makes Perl 5 look like Smalltalk. What’s shameful is its API support. We had to use a very ugly, although apparently common hack to get a DOM of external web pages. This is pure silliness. The browser is already doing the hard work of parsing broken HTML and XML and turning it into a DOM. Why not easily expose that functionality to the developer?    (MEQ)

As Brad dryly noted, “Welcome to my world.”    (MER)

HyperScope Sprint, June 30 in Sebastopol

Last month, we held a HyperScope sprint at Jonathan Cheyer‘s house in San Jose.    (MCC)    (MCD)

It was so productive and fun, we’ve planned another one at the end of this month, Saturday, June 30, from 10am until we drop. This time, we’ll be meeting at Christina Engelbart‘s house in beautiful Sebastopol.    (MCE)

Please join us! This will be an excellent opportunity to learn more about the HyperScope and to hang out with some very interesting and cool people. RSVP at Upcoming or contact me if you’d like to attend, and I’ll forward more details.    (MCF)