The Open Hyperdocument System: An Open System for Collaborative
Knowledge Work
An Open System for Collaborative Knowledge Work
1
Principal Investigator: Douglas C. Engelbart
Co-Investigators: Lee A. Iverson, Eugene Eric Kim, Patrick
D. Lincoln
Submitted to NASA Ames: July 13, 2000 1A
Solving today's most urgent and complex problems, from designing a
spacecraft to managing huge bodies of information, already exceeds the
capabilities of any single person. Further, the complexity and
urgency of many issues facing NASA are rapidly accelerating. Thus,
these issues will soon far exceed the capabilities of current
organizational systems. Matters are made worse by the increased speed
with which key decision makers shift responsibilities, leaving behind
insufficient information for their collaborators to seamlessly
continue. We propose an open evolutionary approach to increasing the
collective ability of groups of people to address complex urgent
problems, through the construction of an Open Hyperdocument
System. 2A
The most important aspect of any collaboration is the sharing and
management of knowledge. There are two opportunities for enhancing
knowledge management in an enterprise such as NASA. First is
improving operations. At a minimum, this entails managing the highly
iterative and dynamic flow of specifications and design information
between an organization and its many suppliers and clients. 2B
Second is the enhancement of NASA's main mission, which is scientific
exploration and discovery. Innovations in these areas require the
ability to manage enormous amounts of data, make it available to
others, and generate knowledge while maintaining the associations with
that data. 2C
Fundamentally, the solution to these problems is the same: a system
that manages the flow of information, the addition of new information,
and the access to and sharing of knowledge. We propose to build a
standards-based, collaborative knowledge management tool called the
Open Hyperdocument System (OHS) that will support deep hyperlinking
between documents. We plan on embracing open standards and building
the Open Hyperdocument System as open source to encourage its
widespread deployment and coevolution. 2D
The Open Hyperdocument System will be an integrated, collaborative
system for storing, editing, viewing, and discussing documents. It
will enable sophisticated linking between all types of documents. The
architecture will be extensible, allowing support for additional
document formats and customizable views. 3A
This system will have four important technical features:
3B
Associating concepts is a natural and powerful method of organizing
information. Hyperlinking provides a way to associate different
concepts, and to easily navigate through recorded knowledge. Of
course, hyperlinking is nothing new. The first hypertext system was
implemented in the 1960s by Doug Engelbart, this project's principal
investigator. Since then, there have been different incarnations of
hypertext systems. 3C
Curiously, the most successful of these systems -- the World Wide Web
-- is probably the least sophisticated. Web hyperlinks are
unidirectional, and the ability to cite documents at a granular level
is limited. These restrictions prevent users from annotating
documents and traversing links in more than one direction. 3D
The Open Hyperdocument System will provide hyperlinking capabilities
that enable sophisticated and desirable features such as annotations
and granular addressability. These capabilities will be built on top
of existing and evolving open standards, such as the W3C's XLink and
XPointer specifications. 3E
Standards such as XLink and XPointer apply only to documents
constructed in eXtensible Markup Language (XML). However, the vast
majority of documents are not constructed in XML, but in a wide
variety of formats, such as plain text, Microsoft Word, and PDF. For
a knowledge management system to be truly useful, it must extend the
capability of legacy document formats to support hyperlinking and
addressability. 3F
By automatically translating documents into an intermediate XML format
without modifying the original document, the Open Hyperdocument System
will support linking to and from legacy document formats. For
example, editors will be able to annotate an article by linking their
comments to Microsoft Word documents submitted by the author. Product
managers will be able to link software requirements documents to
related e-mail discussion threads, use cases, class diagrams, and even
source code. 3G
An extremely flexible and usable user interface is a crucial component
to the system. Not only will it improve users' abilities to access
information, it will help compel people to record their knowledge, one
of the most significant problems in knowledge management. 3H
The Open Hyperdocument System will support customizable views for
documents. For example, when viewing an article, you can have the
system show the entire document or only the first sentence of each
paragraph. Comments and responses to the article can be viewed as
links to separate documents, or they can be displayed within the
article itself. Views can be generated for different types of
devices, from large CRT monitors to braille machines for the sight
impaired. In addition, the Open Hyperdocument System offers an
integrated editing and viewing system. 3I
To support these viewing and hyperlinking features, the Open
Hyperdocument System will be built on top of an extensible application
engine, which will also allow the system to be customized in other
ways. For example, the system could be easily modified to link
certain types of data automatically, or to generate
machine-understandable knowledge representations of documents. 3J
Our primary goals are both to build a highly usable system and to
create an open platform that will rapidly evolve. To achieve these
objectives, we plan on bootstrapping our development effort. Central
to this strategy is deploying this system as open source. On the
surface, this means that the source code will be open and freely
available. However, we believe that this approach will have even more
far-reaching benefits, including the fostering of a large community of
users and software developers using and improving the system. 4A
To facilitate this bootstrapping process, we have decided initially to
target the problem of collaborative software development. While many
of the features that we implement first will be generic enough to
apply to many types of collaborative knowledge work, by creating a
compelling tool for software developers, we can encourage open source
programmers to contribute to our project. 4B
How does the Open Hyperdocument System address the challenges of
collaboratively designing and writing software? Countless studies have
emphasized the importance of the documentation process for the
successful completion and maintenance of software. However, most
engineers spend little effort on gathering requirements, writing
design documents, and annotating source code, preferring instead to
focus on the programming. Accessing knowledge that exists only in the
head of an engineer can be, at best, a chore. Even when this
knowledge is recorded, it is often out of sync with the current state
of the project. 4C
While the documentation process in most software development projects
leaves a lot to be desired, a tremendous amount of recorded knowledge
is produced anyway. The process of designing, writing, and building
software results in many important documents, such as architecture
diagrams and the source code itself. In addition, engineers often
discuss their project over e-mail and other electronic
forums. However, this information is rarely well organized, largely
because of incompatible document formats, poor tool integration, and
poor user interfaces. 4D
Initial versions of the Open Hyperdocument System will address these
issues by providing templates that simplify the documentation process
and by allowing deep linking between related documents, such as
requirements, e-mail discussions, and source code. Customizable views
for these documents will give programmers ways to focus selectively on
the information most relevant to their immediate tasks. 4E
We have divided the project into smaller pieces that we plan on
implementing quickly and releasing aggressively. Our initial targets
for the first year are 4F
Upon completion of these goals, we will have designed and built an
overall framework for the Open Hyperdocument System that can be used
for collaborative knowledge management in all fields. While we
continue to extend the system to support multiple views and other
types of legacy document formats, we will begin to focus more of our
efforts on improving the user interface, so that the system is well
integrated and easy to use. 4G
To maximize the potential for the evolution of this project, we plan
on fully supporting existing open standards while introducing new
ones. For example, one likely contribution will be a specification
for a Distributed DOM (DDOM), which will allow programmers to
manipulate XML documents distributed over different machines. 4H
Initially, our system will be HTTP-based, although that may change as
we continue to develop the system. While the application
infrastructure will most likely be written in Java, we will work hard
to promote language independence and interoperability as much as
possible. 4I
Engelbart, Douglas C. "Augmenting Human Intellect: A Conceptual
Framework." AFOSR-3223 under Contract AF 49(638)-1024, SRI Project
3578 for Air Force Office of Scientific Research, Stanford Research
Institute, Menlo Park, CA. October 1962. 5A
Engelbart, Douglas C. "Collaboration Support Provisions in AUGMENT."
OAC '84 Digest (Proceedings of the 1984, AFIPS Office Automation
Conference, Los Angeles, CA, February 20-22). 5B
The W3C DOM Working Group. "Document Object Model (DOM)." The World
Wide Web Consortium
http://www.w3.org/DOM. 5C
The W3C XML Working Group. "XML Pointer, XML Base and XML Linking."
The World Wide Web Consortium
http://www.w3.org/XML/Linking.
5D