Transfers (by subscribers) use perfect forward secrecy, and can be entirely asynchronous, so sender and receiver need never be online at the same time. Of course if they are, then transmission goes as fast as the physical network allows.

We built the backend and client mainly in Python, supporting Mac and Windows back to 10.5 and XP respectively. For Linux we support 32 and 64 bit Ubuntu 12 and later.

Much of my focus has been on keeping OS variations from impacting the main body of the code. To that end, we're using wxWidgets to abstract the GUI, and I've implemented such native code as needed to interface with unwrapped aspects of the OS and third-party libraries. In some cases the fluent Python bindings to OS facilities just aren't fast enough, and native code is used there also.

There are two problems with connecting a UI to a transaction-based store like the tuplebase. First, how do you efficiently identify the minimal set of interesting changes that have been made in the tuplebase. Second, in reading or writing data, what do you do when a transaction fails to commit?

ZTSoup addresses these problems by maintaining objects locally that record a snapshot of the subset of the tuplebase in which the software is interested. Local changes are recorded in those objects, and they are sent to the tuplebase as a single atomic update operation. It's simply a matter of hooking a call to the soup's Update method into the normal event loop -- all additions and removals of interest in croutons and sieves are sent to the tuplebase, and remote changes to the croutons and sieves are returned.

Because the soup may be talking to a tuplebase on the end of a comms link the update work is split into a synchronous piece, which executes in a small amount of time with no blocking, and an async piece that accumulates work done by the synchronous piece and communicates with the server as fast as latency and server CPU will allow.

But native code can't be included with unsigned applets, so I ported ZTBRep_Client from C++ to Java. ZTBRep_Client is the most useful tuplebase implementation in that it talks to a ZTBServer instance on the end of a comms link, generally a network socket. It's also fairly large, and the async nature of the protocol makes things somewhat tricky. The normal Java synchronization model is based around monitors, which are not directly useful for implementing certain types of concurrent processing, so I implemented condition vars and mutexes to provide more appropriate tools.

New features included support for upload and download of files as attachments to views and notes, headings within views, full text and attribute-based searches, and notification by email. It also supported graphs of useful metrics, implemented across platforms using ZDC_ZooLib.

On the back end, a significant change was moving from the original blockstore implementation to one based on B+Trees that uses a neat upward propagting copy-on-write of nodes to make it virtually bullet-proof - only a physical failure of the backing store can cause data loss.

Language agnosticism is a key feature of tuplebase. Another is its runtime flexibility, which contrasts with most relational database implementations.

The power of a relational database is that the data it stores conforms to a rigorously defined structure. For a single piece of software it's always possible to find a single structure that supports all the work to be performed. As the nature of that work changes the database schema is evolved along with it. But this evolution must be coordinated so that all software that depends on the schema is updated to accommodate the changes, or virtual views are created to preserve a logical interface to the data.

Tuplebase takes a very different approach. Rather than enforcing a structure it allows any structure at all to be stored. Different pieces of software with overlapping concerns obviously must coordinate amongst themselves, but there's no requirement that there be any single authority managing everything. And because there's no mandated structure it's possible at runtime to treat multiple physical tuplebases as a single logical tuplebase, even if they contain data of different provenances. If the code using the data is written to be accommodating to the gradual migration of meaning then newer code can correctly interpret older data, and older code will ignore (but preserve) newer data.

For a time we had native client and web access running simultaneously, but never in a released version. The 3.x client remains buildable as of May 2006, but still talks only to a 3.x server.

The ZooLib file API doesn't address more esoteric file systems like VMS or IBM's MVS. It models a hierarchical system, with accomodations for special roots (like Windows' UNC paths and Mac Classic's multiple volumes with the same name). Files are byte-oriented, and as with other stream operations ZooLib's file Open and Create return a ref counted streamer object, and optionally an error enum providing more detail in the case of failure. The representation of a pointer to a node in a file system also follows ZooLib's practice of a using a smart object with value semantics.

The ZFile suite was the first place I formalized the notion of a 'trail'. This was another case where it was important to come up with a new name that was suggestive of an existing notion, but distinct enough from it that its different semantics could be defined. Picture a tree; a trail is simply the list of steps to be taken to navigate from some node to some other node. A step is either to traverse the link from the node to its parent, or to traverse the link from the node to one of its children. We represent a step as a string. An empty string indicates a 'bounce', a traversal to the parent of the node. Any other string indicates a traversal to the node with that name. Whether the name is attached to the child node, or attached to the link to the child node doesn't matter in this model.

The notion of a trail through a tree shows up in assets and in the handling of HTTP requests. Theoretically it could also be applied to tuples but I'm not quite ready to harmonize everything to that extent.

So, a ZBlockStore takes a single file (or file-like entity) and allows an application to work with resizable blocks allocated within it, easily, efficiently and (with ZBlockStore_PhaseTree) safely.

• Easy to use — blocks are managed with the same API as real files.
• Efficient — all the data lives in a wide fan-out B+ Tree.
• Safe — modifications to nodes in the tree cause the entire ancestor branch to be replicated into existing free space, and the new root is atomically updated only when descendant nodes are known to be in stable storage.

ZooLib abstracts threads, files, networking, graphics and the user interface. And there are escape hatches whereby OS-specific features can be accessed without compromising the platform-agnostic nature of the rest of the code.

ZooLib has been under continual development since 1992, and incorporates fixes for every OS bug or anomaly I've encountered, and generally provides cleaner mechanisms for most tasks than OSes traditionally provide. It's been the basis of Measurement in Motion, NetPhone, and every version of the Knowledge Forum server and client applications.

By this time ZooLib abstracted Mac OS, Windows and UNIX, so the server could be hosted on all three OSes.
The text editing in Knowledge Forum 3.0's note windows used the PAIGE text engine, which was a fairly crusty 60,000 lines by this time, but was portable across Mac and Windows. So although the rest of the client could be built for UNIX, without the text editor it wasn't particularly useful and we never released a UNIX client, although that wasn't a big deal in 2000.
Unfortunately we also had to abandon Kevin Parichan's excellent painting implementation, as it was also Mac-specific. Instead we used ZooLib's NPainter, which used only ZooLib features and was thus completely portable.

CSILE was pioneering and innovative, but in May 1994 limitations in its feature set and a desire to see it in wider use led to a collaboration with Learning in Motion to redesign it. The scope of the necessary changes neccessitated a complete reimplementation and the decision was taken that it would be released as a commercial product.

Cyrus Ghalambor started work on the KF 2.0 client application in July 1995. I was busy with NetPhone till December 1995, when I started working full time on the client's data API and the server that would satisfy its requests. We looked at several existing database libraries, but they were either too limiting or could not run on Mac OS (Classic, of course). So I implemented ZDBase, a portable database engine that was built on ZooLib. Client and server communications and the server's administration UI were also built using ZooLib.

The client application's event loop, menus and windows were handled by a refinement of Berkeley Systems' Zoom framework. It had two main types of windows. The note window used Zoom for its controls, and the WASTE text engine for its text editing panel. The view window provided a two-dimensional icon-based representation of collections of content, its operation was closely modeled on that of Apple's Finder, and it was implemented using ZooLib. The note and view windows also contained pictures, in-line with the text in the note window and in a layer behind the icons in the view window. Pictures were edited in place using a nice painting implementation by Kevin Parichan.

Other dialogs and property inspectors were based on Zoom or ZooLib, depending on whether they were written by Cyrus or by me.

This was my first serious use of B-Trees, the essential basis of most high-performance disk-based storage systems. The searching of B-Tree content and the insertion of new data is well documented in numerous sources, but the deletion of data is invariably left as an exercise for the reader, which was a bit frustrating but ultimately empowering.