Marimba's Castanet product family simplifies the timely propagation and management of software and information across intranets or the Internet. Castanet products can transmit files, Websites, and applications, updating distributed copies automatically or upon user request. Castanet users enjoy the speed and availability of current local copies, while software and information developers benefit from single-site updating. The Castanet infrastructure can propagate updates to user computers around a local area network or around the world. Castanet technology scales from hundreds of users to millions, remaining robust and predictable in the face of network anomalies. It transfers software updates so quickly that daily, hourly, or even more frequent updates are feasible. Castanet security features can control access to sensitive information and ensure confidentiality and integrity when it is transmitted over the Internet.
Castanet products distribute and update two classes of software/information called channels and synchronized folders. Castanet channels can be applications or Websites. Application channels may be written in 100% Java for safety and portability, or for special requirements, in Java supplemented with native methods. Website (HTML) channels may include all World Wide Web media types. Developers publish channels and updates to Castanet Transmitters. Users subscribe to channels by directing a Castanet Tuner to a Transmitter, much as they direct a Web browser to a Web server. A Tuner is a user's channel manager; it lists channels, downloads them, installs them, automatically updates them, and launches them. Tuners are available for Windows 95/NT, Solaris, and Macintosh platforms. Application channels have the performance advantages of conventional applications without the drawbacks of manual installation and upgrading. HTML channels retain the multimedia attractions and timeliness of Web pages along with the network independence and speed that comes from storage on the user's local disk.
Software developers can build the features of Castanet channels into their products with the Castanet UpdateNow library. Such Castanet-enabled products can range from conventional shrinkwrapped applications to software-controlled devices such as telephones. Castanet-enabled products are self-updating; they can transparently download fixes and enhancements over the Internet.
Castanet synchronized folders are implemented by the Castanet UpdateNow products which are available for enterprise use on Windows 95 and NT platforms. A synchronized folder can contain all kinds of files, from price lists to programs, and nested folders. To update a synchronized folder, a user simply double-clicks an icon or runs a script.
Castanet products use differential updating technology to ensure reliable and efficient transfers. This technology is the basis for the Distributed Replication Protocol standard proposed to the World Wide Web Consortium by Marimba, Netscape, Sun Microsystems, Novell, and @Home. For a detailed description see http://www.w3.org/TR/NOTE-drp-19970825.html. Differential updating synchronizes the contents of a client directory (folder) with a server's copy of the same directory, transferring only differences to the client.
Three core products--the Publisher, Transmitter, and Tuner--provide the essential infrastructure for publishing, distributing, and managing Castanet channels. Three auxiliary products--the Proxy, Gateway, and Tuner Administrator--provide special capabilities for particular situations.
Channels are program and data/content files that are stored and updated on servers and mirrored and executed on clients. Running on servers, Castanet Transmitters serve channels to Castanet Tuners which run on clients; the transmission medium is the Internet or an intranet. Tuners and Transmitters work together to automatically keep the channel files on a client consistent with channel updates made on the server. Developers use the Castanet Publisher to assign attributes (title, schedule, and so on) to a channel, to initially copy the channel files from developer directories to Transmitters, and to copy channel updates to Transmitters.
Transmitters, Tuners, and Channels
A channel can be either a Website or an executable program--an application. A Website that is distributed as a channel is called an HTML channel. When a user subscribes to an HTML channel with a Castanet Tuner, the Tuner downloads the channel pages, installs them on the user's disk, and launches a Web browser to view them. The Website is instantly available because it resides on the user's disk. At the same time, an HTML channel retains its currency because the Tuner periodically (or at the user's direction) checks the channel's Transmitter for updates. Differential updating is especially effective for HTML media files. For example, a small change in a large image is transmitted as just the altered bits plus some editing instructions; the Tuner patches the image with the updated bits.
An executable channel is a program that combines:
Channel developers can use the Internet to bring behavior to desktops that is not feasible with traditional means of software distribution:
In each of these examples, differential updating exploits the fact that most of a channel's files rarely change, but a few of them change often. A channel's core code, for instance, rarely changes. In the virtual classroom example, the files that display lessons, submit test results, and establish audio connections between teachers and students are unlikely to change during a course. What will change is the lesson material. Anyone who uses a file synchronizer to update a portable computer or personal digital assistant has experienced the essence of differential updating.
Users obtain channels by subscribing to them with a browser or a Tuner. Each Transmitter has a URL that when visited returns a Web page listing its channels as links. Clicking on a channel link causes the Transmitter to return a subscription which is encoded with a Marimba MIME type. The browser is configured to launch the Tuner as the "helper" application for this MIME type; the Tuner records the subscription.
A channel can be launched from the desktop by double-clicking its icon, by clicking its entry in the Tuner's list of subscribed channels, or by clicking a link to the channel in a Web page. In the last case, the Web browser launches the Tuner as a "helper" which starts the channel.
Each channel has an update schedule that is defined by its developer and can be redefined by users. When the time arrives to update a channel, the Tuner interacts with the channel's Transmitter to obtain the latest version of the channel. A user can also direct a Tuner to update a channel at any time--such as before leaving the office with a portable computer. Allowing for the channel's update interval, automatic channel updating means that users virtually always run the latest version of a channel. If a channel is updated every hour, a connected user's channel is at most one hour out-of-date. A user who suspects a channel is out of date can direct the Tuner to update it immediately.
Because channels are stored on the client disk, they are self-contained. They operate like conventional applications: smoothly and reliably, independent of network or server difficulties, or indeed, whether the client is connected to the network. When the network is available, the Tuner uses it to check for and download channel updates.
Executable channels are written in the Java language and can be supplemented with native methods written in C or a similar language. A 100% Java channel is platform-independent and safe. It is subject to the same run-time scrutiny and prohibitions as a Java applet, except that unlike an applet, a channel can read and write files in a single directory controlled by the Tuner security manager. A channel can use its designated directory to store documents and record information about its state (such as user preferences) for use the next time it is launched.
Some channels need capabilities that are denied to channels by default. For example, a channel that schedules meetings needs access to other computers to inspect their owners' calendars. By default, channels are not trusted to connect to computers except the one that transmitted them. However, a developer can nominate a channel for trusted status, which, if granted by the channel's user, removes all restrictions on the channel's operations. Channels that call native methods must be nominated for trusted status because the Java security manager has no control over what a native method does.
A channel obtained over a company intranet may well be trustworthy, but the same cannot automatically be said for a channel that is downloaded from an Internet Transmitter. The open nature of the Internet enables motivated and clever programmers to deploy programs that can potentially masquerade as Transmitters, and read and alter channel files as they pass by. To thwart potential eavesdroppers and vandals, Castanet channels can be protected by several means. Trusted channels, for example, must be digitally signed by their publishers; a digital signature reliably identifies the publishing organization, and enables a channel to be checked for alterations that may have occurred in transmission. To frustrate potential eavesdroppers, channels can be transmitted from secure Transmitters which scramble the files into a form that can only be decoded by a particular Tuner. These subjects are described more fully in later sections of this paper.
To create or update a channel, a developer uses the Castanet Publisher. To the Publisher, a channel is essentially the contents of a directory called the base directory The base directory is an ordinary directory that contains all of a channel's files--source, executable, HTML, images, and so on. A developer populates a channel's base directory with the programming or Web page development tools best suited for the job. There are no constraints on the contents or location of a channel base directory.
Publishing a channel automatically:
The Publisher can securely publish to a Transmitter located outside a firewall, for example, to a site managed by a third party.
Developers use the Publisher to specify a channel's security properties: whether it is signed and whether it needs trusted status to run. To sign a channel, a developer's organization needs a channel-signing digital certificate. The Publisher includes a facility for managing these certificates, including acquisition, installation, inspection, and selection.
Castanet Publisher (Security Page)
Transmitters serve channels to Tuners; Tuners "pull" channel files from Transmitters. Because performance is critical in Transmitters, they are streamlined and highly optimized. Although multiple Transmitter instances on one machine, or multiple machines, can serve the same channels, repeating transmitters are a simpler way to scale up transmission capacity. Repeating transmitters are described later in this section.
A plug-in is an optional per-channel component that runs on a Transmitter. Its purpose is to receive and respond to feedback data provided by channel instances through the Tuner's BackChannel facility. A channel can accumulate and store feedback data whether or not its user's computer is attached to the network. When the Tuner next requests a channel update, it sends the channel's feedback data to the Transmitter in the update request.
When a Transmitter receives an update request for a channel from a Tuner, the Transmitter invokes the channel's plug-in. Although there is no prescribed recipe that a plug-in must follow, a typical plug-in inspects feedback data sent by the Tuner in behalf of the channel, and either passively records the data or takes positive action. For example, one plug-in might interpret its feedback data as a database request, consult the database, and return the result with the channel update files sent back by the Transmitter. In this example a plug-in/channel pair implements the client-server style of application partitioning. Another plug-in might interpret its feedback data as hints for customizing the channel on the fly; for example, if a channel records the lessons the user has completed, the plug-in would return the next lesson. There are no limits to what a plug-in can return with a channel update. It can send data retrieved from a database, it can congratulate a game user who reaches a scoring threshold (and increase the game's difficulty), it can send a special greeting on an employee's anniversary.
To prevent an unauthorized user from subscribing to channels, a Transmitter can be designated as access-controlled. Such a Transmitter only divulges its list of channels to, and honors subscription requests from, users who can supply valid IDs and passwords. An administrator can maintain a Transmitter's list of authorized users with an editor built into the Transmitter's user interface, or can direct a Transmitter to import a list from an LDAP (lightweight directory access protocol) product.
To ensure Internet users of a Transmitter's identity, and to protect the transmission of sensitive channels from eavesdropping, a Transmitter can be directed to operate in secure mode. A secure Transmitter interacts with Tuners and Publishers under the protection of the SSL (secure sockets layer) protocol standard developed by Netscape Communications Corporation to secure Internet commerce. When a Tuner contacts a secure Transmitter, the Transmitter returns an unforgeable digital certificate which attests to its identity. The Transmitter's user interface includes a facility for obtaining a certificate. If the Transmitter's certificate is in order, the Transmitter and Tuner then agree on a key to use in this session. The Transmitter uses the key to encrypt the channel's files, and the Tuner uses it to decrypt them. The encrypted files are useless to an Internet eavesdropper who manages to intercept them. Castanet certificates are provided by VeriSign Corporation, and encryption technology is from RSA Data Security, Inc.
Castanet Transmitter (SSL Certificate Management Page)
To increase transmission capacity without adding administrative burden, a channel's Transmitter can be supplemented with repeating Transmitters, called Repeaters for short. A channel developer uses the Publisher to list the Transmitters that are to act as Repeaters for the channel. Whenever the channel is published, the Publisher automatically updates the Repeaters as well as the channel's primary Transmitter.
A channel's Repeaters can be located at the same site as the channel's primary Transmitter, or they can be geographically dispersed. Each Repeater can have its own local disk, eliminating the network traffic that would result if transmission capacity were increased by using multiple Transmitters sharing a network-mounted disk.
When a Tuner subscribes to a channel that has Repeaters, the Transmitter automatically redirects the Tuner to a Repeater. The assignment strategy is specified by the channel developer: either round-robin (which spreads the load evenly among Repeaters) or geographical (which chooses the Repeater in the time zone nearest to the Tuner). Having been assigned to a Repeater, the Tuner obtains updates from that Repeater, off-loading the Transmitter. However, the Tuner remembers the address of the original Transmitter; if a Repeater is down, the Tuner returns to the Transmitter to obtain the address of a substitute Repeater. By adding local and remote repeaters to a Transmitter, channel providers can scale transmission capacity from a hundred subscribers to a million, providing fast response anywhere in the world, with no more difficulty than publishing to a single Transmitter.
A Tuner is a user's channel manager; it lists Transmitter channels, downloads and installs them, updates them, and provides a user interface to security features such as signed and trusted channels. Tuner updates are driven by a per-channel schedule that is initialized by the channel's developer and modifiable by the channel's user. One channel's schedule may direct the Tuner to check for updates every month, another may specify hourly updates.
Castanet Tuner
A channel update can be performed while the current version of a channel is running. In such a case, the Tuner notifies the channel that it has new files. The channel may ignore the notification, in which case the Tuner defers the update until the channel quits. Alternatively, the running channel may recognize the new files as data, ask the Tuner to install them, and then read the files. Channels can use this "hot update" technique to display incoming news or any other data that users should see immediately.
Tuners download and update channels under the protection of transactions, ensuring that channel files are always self-consistent. There is no chance, for example, that version 4.1 files will be mixed in with version 4.0 files, as can happen when Web browers mix cached applet files with newer downloaded files.
Tuners are pre-subscribed to two channels. The Introduction channel is launched automatically when the Tuner is installed, and gives a five-minute summary of Tuner usage. The Channel Guide Channel has its own button on the Tuner. It is a "channel of channels," a categorized and annotated catalog of channels selected by Marimba. Users can subscribe to channels listed in the Channel Guide without having to know the name of a Transmitter.
A Tuner not only updates channels, it is one. Tuners normally check weekly for updates to themselves from Marimba Transmitters, but they can also be customized with the Tuner Administrator (described later in this paper) to get their updates from a Transmitter designated by an administrator.
When a Tuner connects to a secure Transmitter, it validates the Transmitter and decrypts the files it receives without user intervention. The Tuner brings other security-related events to the user's attention. In particular, if a user subscribes to a channel whose developer has nominated it for trusted status, the Tuner notifies the user, presents the developer's digital certificate, and asks the user to either grant or deny the developer's request for trusted status. If, having inspected the certificate to learn who developed the channel, the user concurs, the Tuner launches the channel. If the user denies the trusted status request, the Tuner keeps the channel in case the user wants to perform further investigations, such as asking a colleague about the channel. The user can delete the channel without ever launching it, and can revoke a trusted channel's status at any time. The Tuner verifies the integrity of signed channels when they are downloaded and also when they are launched; the latter check detects damage to a signed channel that occurred on the user's disk. Signed channel users know that the channels they launch are bit-for-bit identical with the channels that were published by their developers.
A Castanet Proxy preserves the protective properties of a corporate firewall without creating a bottleneck. Because a Castanet Proxy is dedicated to serving channels, it can act more intelligently than a typical caching HTTP proxy. Situated in a firewall, a Castanet Proxy passes Tuner requests to Transmitters so channel plug-ins can process feedback data and customize returned channel files. A Proxy also caches copies of files obtained from Transmitters to minimize cross-firewall traffic. Finally, a Proxy optimizes connection utilization by sending multiple update requests in the same connection.
Castanet Proxies Pass Tuner Requests and Cache Channel Files
Tuners inside the firewall are configured to send their requests to the Castanet Proxy instead of to Transmitters. When a Proxy receives a channel update request from a Tuner, it connects to the Transmitter on behalf of the Tuner and forwards the Tuner's update request. Forwarding Tuner requests ensures that channel plug-ins receive feedback data as if no Proxy were present. For each update request, the Transmitter returns a list of the current channel files--as influenced by the plug-in, if there is one--but no files. The Proxy compares the Transmitter's file list with its cache, obtains missing files from the Transmitter, caches them, and returns the necessary files to the Tuner. Proxies keep connections to Transmitters open so that multiple requests for the same channel avoid the overhead of establishing a separate TCP connection.
To appreciate how Castanet Proxies minimize cross-firewall traffic, consider an example. Suppose 1,000 users inside a corporate firewall subscribe to a news channel which includes a file that's updated hourly. With no Proxy, every hour the Tuners would request updates from the news channel Transmitter, causing the same file to be transmitted across the firewall 1,000 times. A simple-minded caching proxy (along the lines of an HTTP proxy) would reduce the cross-firewall traffic but would deprive the news channel's plug-in of feedback data. A Castanet Proxy cuts cross-firewall file transfers while keeping the channel plug-in "in the loop." Moreover, the cost of including the plug-in is minimal because all 1,000 requests are passed in the same connection. In sum, a Castanet Proxy dramatically reduces the load on a firewall machine while preserving the channel-Transmitter semantics that prevail when no firewall is present.
Sometimes, especially when a Transmitter and a Web server are protected by the same firewall, administration would be simplified if the Transmitter and Web server used the same host and port. Although port sharing is not possible, a Castanet Gateway does the next-best thing by making a Transmitter appear to share a Web server's port.
A Gateway is a Web server plug-in that is invoked when a Transmitter request arrives on the Web server's port. The Gateway forwards the request to the host and port where the Transmitter is really running.
The Castanet Tuner Administrator enables companies to distribute customized Tuners within an enterprise. The Tuner Administrator can create an installable Tuner which differs from Marimba's standard Tuner in any of these ways:
Changing the Tuner's Introduction and Channel Guide channels enables a company to present Tuner users with the company's version of "getting started" and descriptions of the company's channels.
Changing the Transmitter from which the Tuner updates itself gives the company control over Tuner updates. Tuner updates originate as bug fixes and enhancements made by Marimba and published to a Marimba Transmitter; standard Tuners check this Transmitter for updates each week. Although not likely, it's possible that a company channel is incompatible with a new release of the Tuner. To prevent any disruption from such an incompatibility, an administrator can:
Because the company Tuners update themselves from the company Transmitter, they will only absorb changes that a company administrator has verified and published.
Software developers can use the Castanet UpdateNow SDK to create Castanet-enabled applications in Java (optionally supplemented with native methods) or in Windows 95/NT C/C++. A Castanet-enabled application can have the same properties as a Castanet executable channel, including:
Unlike channels, Castanet-enabled applications are independent of Tuners. They can be distributed in shrinkwrapped boxes, or embedded in "appliances" such as telephones. Nor does a Castanet-enabled application need a Tuner to obtain updates from a Transmitter or send feedback data to it. The relevant Tuner operations are built into the UpdateNow SDK. The application connects to the Internet and calls SDK methods or functions which interact with the Transmitter in behalf of the application.
Enterprises can apply Castanet synchronizing capabilities to non-channel data and software with the three Castanet UpdateNow client applications. The UpdateNow clients use Castanet technology to synchronize designated Windows 95/NT user folders on demand. A synchronized folder can contain any combination of code files, data files, and subordinate folders. Because a synchronized folder can be updated with anything, the UpdateNow clients are appropriate for enterprises that have administrative control over the synchronized material.
The master copy of a synchronized folder is published to a Transmitter exactly like a Castanet channel. A tiny user file, called an update file, contains the specification for synchronizing a folder. When a user double-clicks on an update file, the UpdateNow User application saves the current state of the folder, sends an update request to the folder's Transmitter, receives the differential updates, and installs them--making the user's folder an exact copy of the master folder on the Transmitter. To update multiple synchronized folders in one operation, or to schedule automatic updates, update files can be launched in scripts.
The UpdateNow User application saves the previous generation of a synchronized folder in case there's a problem with the new version. For example, the new version of the master file might be inadvertently published with an incorrect file or a program that has a bug. The UndoNow application restores a synchronized folder to its previous state, giving users a fallback option when errors occur. The UndoNow application can also be used to remove changes a user has accidentally made directly to a synchronized folder.
The UpdateNow Administrator application is a superset of the User application; it can create update files as well as run them. To centralize control over update files, a company can restrict the distribution of the Administrator application. An administrator can create update files and distribute them as email attachments. Users can copy attached update files to their desktop or folders, or can simply double-click them to run them when they arrive.
The UpdateNow User and Administrator applications can communicate with secure and access-controlled Castanet Transmitters, and can download signed folders, displaying the signer's certificate for the user's verification.