Staying in the Loop: Multicast Asynchronous Notification for Intranet Webs

W. Scott Meeks, Charles L. Brooks, Frederick J. Hirsch

The Open Group Research Institute, 11 Cambridge Center, Cambridge MA 02142, USA


The Web at present does not easily support the passive receipt of information in a timely manner. This paper describes the integration of a notification system with other Web tools. A Tcl-based toolkit using the MIT Zephyr Notification System is used to allow applications to notify each other or a user of relevant information or changes in information. A blackboard service is also used to temporarily and selectively store the information in notifications for later use by applications. The use and effectiveness of the toolkit and service are demonstrated in a simple application that may be used to conduct a distributed guided tour of individual Web pages. Notifications are also integrated into our Mediator and HistoryGraph tools to inform users of updates in a timely manner.

Keywords: Notification, Blackboard, Tools, Application Program Interface

1. Introduction

Although a number of tools exist for a user to actively find information on the World Wide Web, the current Web structure does not easily support passively receiving information in a timely manner. The focus of the research described below is on extended web-based tools and services that allow the information to find the user, rather than the reverse. These tools serve the user who wants information on a given topic delivered as soon as it becomes available (without having to aggressively search for it or filter through an abundance of irrelevant information,) and supports network services that wish to report on significant events.

This paper explores how notification can be used to to achieve these goals. We use notification here to mean the transport and delivery of an important announcement or piece of information to interested parties. As used here, notification is unidirectional, self-contained, and can occur between any combination of users and applications. We have developed a toolkit that provides a programmatic interface for sending and receiving notifications. Additional services have been developed using this toolkit, including a simple network blackboard system where remote applications can share data, and a notification proxy service (built on the blackboard) that an application can use to temporarily receive notifications when it is unable to do so itself.

Our research is focused on the workings of groups and teams within an Intranet Web. The term Intranet is generally used in contrast to Internet to indicate an Internet-based network that is focused on, or restricted to, a particular company or organization. The term "Intranet Web" is used here to indicate a portion of the Web that is completely controlled by a particular company. Some or all of this portion may be visible from outside the company and users within the company may have access to the rest of the Web, but access and administration are completely controlled from inside the company. Remote access is supported for authorized users.

We present a brief scenario of how notification might be incorporated into the use of the Web within an Intranet context. Goals and requirements are discussed in detail, followed by the design and implementation of the approach. Finally results are reported and ideas are discussed for enhancing the notification tools in the future.

2. Scenario

A simple scenario illuminates and motivates some of our work. It uses the following tools, described in more depth later in this paper:
A Web visualization prototype. It automatically tracks the user's browser's navigation actions, building up an interactive tree structure of the user's paths through the web. It provides organization (in the form of sets of nodes) and restructuring facilities, and, via the notification mechanisms, can also display information about events in the Intranet Web. [Hirsch97]
A document access mediation service. This service provides controlled access to a set of structured documents for a team of users. Functions include creating and updating documents; change notifications are automatically multicast to subscribed users.
A program for providing "guided tours" of the Web. One individual acts as a "tour leader" to provide a "guided tour" of a particular Web space; tour members "follow along" via their browsers.
Our Tcl toolkit providing a programmatic interface to a notification system.
Intranet Blackboard
A server providing a location independent mechanism for storing and retrieving information. A notification recording and retrieval service (used by HistoryGraph) is built on top of the blackboard and allows it to act as a notification proxy for users.
Event Manager
A proposed tool that allows more sophisticated user control over the disposition of notifications received by the user.
Bill works in the Boston office of a moderate sized software company with a few additional locations across the country. He comes into his office one morning and, as usual, checks his HistoryGraph window to see if new notifications have arrived. He's configured his HistoryGraph to automatically put notifications into a set called "New" in addition to whatever other set they are added to. He can quickly scan through this set to see new notifications and then clear the "New" set.

This morning he sees that the node for the team seminar schedule has a notification (multicast to everyone who has subscribed to seminar notifications) from the Mediator attached to it, indicating that it has been updated. He activates the node to bring the document into his browser, and sees that Janet's talk at the New York office has been rescheduled from this morning to after lunch. He had planned to attend this talk "remotely"; the audio and slides will be available via the Web using the GroupWalk program to synchronize.

Bill returns from lunch a few minutes late. He assumes that Janet is running the GroupWalk program in tour guide mode. He starts his GroupWalk program in tour member mode; it retrieves the URLs of the the previous slides Bill missed and jumps his browser to the current slide. This allows Bill to use the browser's back button to view the previous slides. As Janet continues to give her talk and advances through her slides, Bill's browser follows along.

Bill thinks this is an acceptable method for attending a talk remotely but still doesn't compare to being there. This thought reminds him that he is leaving for a conference that evening and needs to prepare. In particular, he's been working on a document that is due while he is gone. He's not quite done with it, so he needs to get Bob to finish it. He uses the Mediator to delegate control of the document to Bob, adding a short message to the notification indicating what Bob needs to do. Bob's HistoryGraph will receive this notification and add it to the node for the relevant document (adding the node itself if necessary.) In addition, Bob has configured his Event Manager so that Mediator notifications listing him as the delegate will pop up a window on his terminal displaying the notification.

Since this document is very important, Bill wants to be able to monitor Mediator notifications while at the conference. However, he'll be detached from his company's Intranet while traveling and possibly at other intermittent times depending on the available connections at the conference. Thus he would miss any notifications sent out while disconnected. However, Bill's HistoryGraph provides a facility for storing his notifications until he can retrieve them later.

Bill invokes the "Record Notices" function and then quits out of the HistoryGraph. Whenever he has some free time at the conference and is able to connect to his Intranet, he can invoke "Retrieve Notices" to get the newest Mediator notifications. He thus is reassured as he sees Bob making regular updates to the document and then sealing it in it's final form well before the deadline.

3. Goals

Asynchronous Notification of Changes in the Web

Our primary goal was to provide a simple and extensible system for sending and receiving asynchronous event notifications between Web tools as well as to users. In this case, asynchronous really means a) not directly tied to browser/server interactions (so that server push, browser pull, or proxy additions to documents were not sufficient) and b) potentially delivered and acted on as soon as possible.


We decided on a system which uses selective multicasting so that we could support dynamically changing groups of users and tools in a scalable manner. We also wanted users and tools to only receive notifications of interest to them.

Location Independence

Based on the mobility of both users and services running on the network (not to mention possible changes in network configurations) we desired that a notification could be delivered independent of the actual location of the entity receiving it. Indeed, given that both users (logged in at the office and at home) and services could have multiple points-of-presence on the network, we wanted notifications delivered to all instances of a receiver.

Persistent Storage

In addition to having important pieces of information delivered in a timely manner, we wanted to provide persistent storage of notification information than can be easily shared among our tools. In essence, we wanted something like a blackboard that could serve both as a message repository and as a rendezvous service. This is particularly important in light of services that are not always running, or may have crashed and restarted, and for users who may have temporarily disconnected from the Intranet.

4. Our Intranet Workshop Environment


We liken our Intranet environment to a workshop where each user has their own station and own tasks, but where these stations and tasks are interlinked in various ways. Within this environment, we divided our Intranet services into three general classes: desktop applications, group services, and communication services.

Our desktop applications consist primarily of what we call browsing associates. A browsing associate is a small and simple application (compared to browsers and servers) which is not coupled to a particular HTTP stream and can independently and asynchronously access the Web on the user's behalf. Our associates often take advantage of browser interfaces to observe user browsing actions, but this is not a requirement. These applications include our HistoryGraph [Hirsch97] associate for tracking browser navigation history, and our WhatsNew[Brooks95] associate for monitoring changes in the Web.

Our group services include the Mediator [Mediator] for controlling access to a set of structured documents for a team of users. Mediated documents are pages which have owners and controlled access. The Mediated access service provides an index of documents as well as producing the original documents to authorized users when requested. The group and graphical annotation services[Schickler96] are used to add inline annotations to Web documents without modifying the actual document. The user profile service stores common user information shared by the other services.

[Notification Architecture]

Figure 1: The Notification Software Architecture and its Use.

Figure 1 shows how the notification software is structured and how our other applications and services relate to it. Our communication services are based on two subsystems. The first is a notification package that links with the Zephyr Notification System [DelaFera87], developed as part of MIT Project Athena. This package (written in Tcl) provides a higher level abstraction layer for automatically subscribing and unsubscribing to notices, for handling notices as they are received, and for synchronizing responses to notices. It also provides the ZSend protocol, a display and location independent replacement for the send primitive in Tcl/Tk which is used to communicate between different applications.

The second subsystem is an implementation of the Linda programming language that uses a TCP/IP-based server to represent a shared tuplespace [Schoenfeldinger95]. A tuple is a list of fields containing specific values; other than underlying system limitations, there are no limits on the number or length of fields. The Linda language defines a set of functions to access and modify tuples.

The blackboard service consists of a notification-aware (and thus location independent) front-end server for the Linda server. The naming service provides a central registry for generating unique but meaningful application names. It is built on top of the blackboard service and (if running) is used by the notification toolkit. The notification recording service allows the blackboard to act as a surrogate for another application that is temporarily disconnected from the network, storing notices for later delivery when the application reconnects. This service uses both the notification toolkit and the blackboard service.

Adding these facilities to existing applications was straightforward. WhatsNew uses just the ZSend protocol; adding this functionality required one new function call and globally substituting the names of two other functions. HistoryGraph makes full use of the notification facilities: it uses ZSend to communicate with WhatsNew, responds to notifications from the Mediator service, and uses the blackboard notification recording service to record and retrieve Mediator notices sent while HistoryGraph is not running. The changes made to HistoryGraph were significant but required few code changes, consisting mainly of adding calls to various initialization routines, adding routines to handle incoming notices and returns from the blackboard, and minor user interface changes to reflect the enhanced capabilities.

The Mediator sends out email notification whenever a change is made in any document under its control, so the only change was to add a call to the zwrite command (part of the standard Zephyr system) to also send the information as a notification. The other group services do not yet make use of the notification facilities.

The MIT Zephyr Notification System

When designing our communication facilities, we had to consider whether to build our own underlying transport mechanisms from scratch or to base our work on some other system. After examining various possibilities (see below), we felt the MIT Zephyr system best met our needs.
Zephyr is a multicast notice transport and delivery system, ... for use by network-based services and applications with a need for immediate, reliable, and rapid communication with their clients. [...] It is designed as a suite of "layered services" based on a reliable, authenticated notice protocol. Multiple, redundant Zephyr services provide basic routing, queuing, and dispatching services to clients that communicate with the Zephyr client library. More advanced services are built upon this base.[DelaFera87]

We chose to exploit several capabilities in the Zephyr notification system:

Alternatives to Zephyr

Before deciding on Zephyr as our notification transport mechanism, we examined three other systems: the Active Map Service, the Information Bus, and the Pilgrim Event Notifier.

The Active Map Service (AMS)[Schilit94], developed at Xerox Palo Alto Research Center, supports context-aware computing by providing clients with information about located-objects and how those objects change over time. Most relevant to our work are the subscriptions, which are standing queries about the locations of objects, and the use of multicast channels using IP multicast datagrams. Zephyr has two advantages for our work: it's a robust, deployed system and it doesn't rely on IP multicast which is not yet widely supported.

The Information Bus (IB)[Oki93], developed by Teknekron Software Systems, is an extensible, distributed communications system operating in an environment that requires continuous operation, dynamic evolution and integration with extant systems. This would have been an ideal system if it weren't for the following limitations:

The Pilgrim Event Notifier (PEN)[DiBella94] is part of Project Pilgrim at the University of Massachusetts, Amherst. PEN was inspired by Zephyr, but based on the Open Software Foundation's Distributed Computing Environment (DCE)[OSFDCE] rather than UDP and Kerberos. We decided to go with Zephyr because of its wider availability, but PEN would be a likely alternative in a DCE environment.

Toolkit Implementation

Our notification toolkit is implemented in Tcl[Ousterhout94][Welch95]. It consists of a loadable Tcl package containing the bindings to a subset of the Zephyr library routines and a Tcl library of higher level routines built on top of the Zephyr primitives. We chose Tcl as the implementation language for several reasons:

Blackboard Implementation

Our blackboard facility is implemented on top of the Linda server. The match between the Linda notion of tuplespace and the Zephyr definition of a notice as a 4-tuple meant that Zephyr notices easily could be stored into the tuplespace by utilizing the existing <class, instance, recipient> fields, adding a time-stamp, and encoding the data field in a standardized way (we chose the encoding scheme used by the HTTP post primitives).

The box labeled Blackboard in Figure 1 really consists of three parts:

  1. a Zephyr-enabled front-end server that connects directly to the underlying Linda server and implements the blackboard protocol(2) and library(3),
  2. a protocol that specifies the format of the Zephyr notices exchanged between the front-end server and an application to implement the basic blackboard functions,
  3. a Tcl library of functions that provide applications with the abstract interface to the blackboard.

The Naming service is built on top of the blackboard service. Clients of this service request registration of a particular name; the service checks for instances of the requested name in the blackboard, adds a unique number to the end of the name if it's already in use, and returns the assigned name to the requester. The notification toolkit itself uses the Naming service in implementing the ZSend protocol.

The Recording service uses both the notification toolkit and the blackboard service. A client issues a request that notifications of a particular class and instance be recorded. The service uses the toolkit to subscribe to the notifications and uses the blackboard to store the notifications, along with the name and user of the application. Later the client can retrieve recorded notifications and stop recording new ones. The recording service queries the blackboard for the relevant notifications and uses the toolkit to unsubscribe to further notifications.

The implementation of the blackboard service provides the following improvements over using Linda alone or some other database-like mechanism:

  1. Location independence: the service does not have to live at some well-known location. Applications need not be configured to go to some special host or port. Indeed, the services itself can be moved or replicated as needed.
  2. Simplified interface: the Tcl blackboard library provides a simplified interface to the underlying Linda server, access to a simple application naming service, and (eventually) support for more powerful abstractions.
  3. Naming service: by providing a central registry for generating unique but meaningful application names, the blackboard supports services like the ZSend protocol and eventually for a service brokering system.
  4. Application proxy: the blackboard can act as a surrogate for an application by capturing information for that application while it is off-line. For example, HistoryGraph can use the blackboard to save Zephyr notices from the Mediator while it is off-line and then retrieve these notices when it comes back on-line.

5. Results

We have developed two applications utilizing the notification toolkit and blackboard service. The first is a "guided tour" of individual Web pages. The second is the integration of the Mediator and HistoryGraph using notification and the recording service.

The GroupWalk Associate

The GroupWalk associate takes on two roles. In one role, it watches the tour leader's browser and sends out a notification whenever the current URL changes. In the second role, it receives these notifications and tells a tour member's browser to fetch the correct URL. Should a tour member arrive late for the tour, they can "catch up" by automatically retrieving all pages up to and including the current page in the order accessed (thus enabling sensible usage of the browser's Back button).

The prototype implementation of the GroupWalk program worked well. Within our local network, side-by-side demonstration of a leader and multiple tour members showed that, even at a fairly fast pace, the tour member's browsers remained synchronized with the tour leader well within lag times typical of the Web, on the order of a second at most.

The initial implementation only supported running a single tour at a time; if more than one leader program runs at the same time, all the tour members receive the notifications for all the tours together. (The toolkit can support uniquely naming tours so that multiple tours could run simultaneously.)

Integration of the Mediator and HistoryGraph

Notices from the Mediator are incorporated into the HistoryGraph by modifying the icon representing a given document, thus providing a visual cue of changes in the mediated namespace. In addition, all mediated documents are placed into a separate set as they are discovered; this allows scanning through all known mediated documents and performing operations on all these documents at once.

Figure 2 shows an example of what our HistoryGraph associate looks like with additions specific to the notification work. The contents of the Mediator menu are being shown. This menu does not normally appear in the HistoryGraph, but will be automatically added if the user has enabled the use of notification and the blackboard and if HistoryGraph detects the presence of a blackboard in the network. The Record Notices item tells the blackboard to save notices from the Mediator for later retrieval by this user; Retrieve Notices retrieves recorded notices from the blackboard and incorporates them into the HistoryGraph display.

[HistoryGraph Showing Mediator Additions]

Figure 2: Sample HistoryGraph Screen Showing Mediated Documents and the Mediator Menu

The main HistoryGraph display consists of nodes representing visited URLs and links representing the order in which URLs were visited. The nodes consist of small icons followed by an elided title or URL. The standard icon is a simple file folder icon and indicates no additional information about the page. Additional icons are used for nodes with additional information. Three new icons were added to support notification. The document icon ([Icon of Page with Lines]) indicates that the document is stored in our Mediated access service, and the stack of documents icon ([Icon of Stack of Pages]) indicates that the document is an index of Mediated documents. The "sealed" icon ([Icon of Circle with Dot]) indicates that the document has been sealed and can no longer be modified. (The icon is meant to represent an old-fashioned sealing wax seal.)

Notification Toolkit

The initial Zephyr package was simple to implement: Tcl has a powerful facility for binding Tcl procedures and data to C libraries. The Zephyr library is fairly low level, dealing primarily with the details of sending and receiving raw notifications; the major effort was designing and implementing higher level abstractions that would simplify the work done by the associate and service programmer.

6. Futures

Enhanced Notification Facilities

Integrating notification mechanisms into the Web is a critical element of groupware strategy and an important research area in the Web community. There are two different methods for determining which users need to be notified when changes occur in a mediated portion of a web:

The object-oriented notification has the benefit of only notifying a user when that particular object changes; however, the user can miss changes to other (related) documents. The subscription-list approach puts the onus on the end-user for filtering the potentially large number of notifications: this might well-be a job for a browsing associate. It also potentially relies on someone else putting together useful and directed lists. An interesting alternative is to represent subscription lists as individual objects: this allows the user to examine the lists to determine their properties and to receive notifications if these properties change, such as by adding or removing a subscriber.

We are considering a general-purpose Event Manager that would include the functionality currently supported in zctl (Zephyr subscription list control application) and zwgc (Zephyr windowgram client - responsible for displaying notifications). Users would be able to graphically view and manipulate their subscription lists, and associate actions with each element. Each action could be specified via a simple scripting language based on Tcl or Expect. Ideally, we would support additional forms of notification as well as escalation and expiration.

Escalation increases the importance of a notification, generally in response to external factors. These factors may be simple (such as the passage of time or the approach of a deadline), or may be more complex (dependent on the role of the user or associate receiving the notification). For example, a printer might multicast a notification that it's jammed; most users receiving the notification would handle it as low priority; anyone actively printing might have it escalated to medium priority; the person in charge of clearing printer jams would have it escalated to high priority. The escalation of a notice can be reflected in a number of ways: it may have increased visibility, it might be repeated periodically, or it may require an acknowledgment before going away.

Expiration is the time after which a notification is invalid, similar to the expiration date on most coupons. The primary use of expiration would be to automatically reprocess notifications after the expiration time passes, such as by refiling or discarding them. Expiration time could also be one of the factors handled by escalation.

Enhanced Blackboard Facilities

The blackboard service serves two main purposes: a message repository and a rendezvous service for cooperating associates. We have demonstrated how an associate can invoke the blackboard to record notifications of interest, and then retrieve these notifications once the associate is re-started. Since the blackboard can always be located via its Zephyr interface, co-operating agents (either working together on a single desktop, or co-operating across user's systems) can register their presence as well as the capabilities they provide. In addition, associates can choose to synchronize behavior via the underlying Linda primitives. Associates thus have both a synchronous and asynchronous method for process co-ordination.

Additional Features

Integrate Annotation Notifications
Our annotation system merges inline annotations into a source document to generate a new, annotated document. As with our Mediator system, we foresee modifying the annotation system so that it sends notifications based on the addition or modification of annotations. The HistoryGraph or some other associate would be modified to receive and display these notifications.

Expand Mediator Notifications
Currently, the notifications from the Mediator do not contain all the information that the Mediator has available (in particular, they do not include the log message for document modifications), and the HistoryGraph does not save or display all the information in the current notifications.

Additionally, by tracking the state of the document, the state change itself can be used to trigger other activities. For example, the sealing of a document (no further modifications are permitted) could trigger the movement of that document into a "read-only" area of the server or could cause access rights to be given to additional individuals. These actions could be performed by other associates acting on notifications from the Mediator.

7. Conclusions

We set out to build tools to support passively receiving information from the Web in a timely manner. One approach was to build a notification toolkit based on the Zephyr system and a blackboard service based on the notification toolkit and the Linda tuplespace server. To demonstrate the value of this software, we built the GroupWalk associate and extended our HistoryGraph associate to respond to Mediator notifications.

The notification toolkit and blackboard service proved to be remarkably powerful and effective for a number of reasons. First, Zephyr provides a flexible yet practical notification infrastructure. It is limited by administration requirements and its local area focus, but this matches our Intranet Web focus. The use of directed multicast simplified many potential routing and location problems.

Second, the blackboard service proved effective for making the notification service itself more robust, for temporary storage of shared associate information, and for establishing an application proxy for notifications. It can also be used by associates and services to synchronize with other associates and services, though asynchronous communication alone can support synchronization within an approximate interval (around one second for our applications which seems typical of the latency of the Web). Ultimately, we were able to provide a flexible building block for other systems. The more we used it, the more we thought of uses for it.

The generation of notifications when Web based resources change state easily leads to the perception of these notification as events. Enabling users to respond to these events (using capabilities such as notifications or email messages) enables a view of event-based interaction that echos the event-based implementation of many Graphical User Interfaces: Notification in the context of the Web thus becomes "the event loop in the large". This notion has two major consequences:

We believe that our "Intranet workshop" notion provides a base for building Web-based information systems that can combine the ubiquity of the Web user-interface with a flexible and powerful infrastructure for sharing information about the state of on-going project work.

Availability of Software

The notification toolkit, blackboard, Mediator, HistoryGraph and WhatsNew are available at . The GroupWalk associate was a very early prototype and a version for external release has not been produced.

8. Acknowledgements

This research was supported in part by the Defense Advanced Research Projects Agency (DARPA) under the contract number F19628-95-C-0042. The views and conclusion contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or the U.S. Government.

9. References

[Brooks95] C. Brooks, W. S. Meeks, M. S. Mazer, An Architecture for Supporting Quasi-agent Entities in the WWW, Intelligent Agents Workshop Proceedings, Conference on Information and Knowledge Management, December 1995.
[Cygnus] The GNU-Win32 Project Page,
[DelaFera87] C. A. DelaFera. The Zephyr Notification System. MIT Project Athena documents, Massachusetts Institute of Technology, Cambridge, MA, USA, 1987.
[DiBella94] K. DiBella, PEN: Event Notification in Distributed Environments, UMass - Project Pilgrim, August 1994.
[Gutfreund94] Y. Gutfreund, et. al., WWWinda: An Orchestration Service for WWW Browsers and Accessories, The Second International WWW Conference `94: Mosaic and the Web, Advance Proceedings, October, 1994.
[Hirsch97] F. Hirsch, W. S. Meeks, C. Brooks, Creating Custom Graphical Web Views Based on User Browsing History.
[Mediator] Distributed Authoring with the Mediator,
[Miller87] S. Miller, C. Neuman, Kerberos, MIT Project Athena Technical Plan, section E.2.1, 1987.
[Oki93] B. Oki, M. Pfluegl, A. Siegel, D. Skeen, The Information Bus--An Architecture for Extensible Distributed Systems, Proceedings of the Fourteenth ACM Symposium on Operating System Principles, pps. 58-68, December 1993.
[OSFDCE] OSF DCE User's Guide and Reference, Prentice Hall, 1993.
[Ousterhout94] J. K. Ousterhout, Tcl and the Tk Toolkit, Addison-Wesley, 1994.
[Schickler96] M. Schickler, M. Mazer, C. Brooks, Pan-Browser Support for Annotations and Other Meta-Information on the World Wide Web, Computer Networks and ISDN Systems 28 (1996)1063-1074, Fifth International World Wide Web Conference.
[Schilit94] B. Schilit, M. Theimer, Disseminating Active Map Information to Mobile Hosts, IEEE Network, pps. 22-32, Sept./Oct. 1994.
W. Schoenfeldinger, WWW Meets Linda: Linda for Global WWW-Based Transaction Processing Systems, World Wide Web Journal, Issue 1: Conference Proceedings, Fourth International World Wide Web Conference, O'Reilly and Associates, December 1995.
[Welch95] B. B. Welch, Practical Programming in Tcl and Tk, Prentice Hall, 1995

W. Scott Meeks, Open Group Research Institute. Email:

Charles L. Brooks, Open Group Research Institute. Email:

Frederick J. Hirsch, Open Group Research Institute. Email: