« Earlier Projects | Electric Magic Portfolio | Newer Projects »

October 2002

Tuplebase

ZooLib's tuplebase is derived from the tuplespace concept initially explored in the Linda coordination language, another well known derivation of which is Sun's JavaSpaces system. Whereas JavaSpaces is Java-only and relies on many of that languages's features, ZooLib's tuplebase works today with C++ and Java, and is well suited to work with other languages.

Language agnosticism is a key feature of tuplebase. Another is its runtime flexibility, which contrasts with most relational database implementations.

The power of a relational database is that the data it stores conforms to a rigorously defined structure. For a single piece of software it's always possible to find a single structure that supports all the work to be performed. As the nature of that work changes the database schema is evolved along with it. But this evolution must be coordinated so that all software that depends on the schema is updated to accommodate the changes, or virtual views are created to preserve a logical interface to the data.

Tuplebase takes a very different approach. Rather than enforcing a structure it allows any structure at all to be stored. Different pieces of software with overlapping concerns obviously must coordinate amongst themselves, but there's no requirement that there be any single authority managing everything. And because there's no mandated structure it's possible at runtime to treat multiple physical tuplebases as a single logical tuplebase, even if they contain data of different provenances. If the code using the data is written to be accommodating to the gradual migration of meaning then newer code can correctly interpret older data, and older code will ignore (but preserve) newer data.

October 2002

ZDC_ZooLib: Portable Graphics

ZooLib defines and implements a graphics API that produces identical results across all supported platforms. It is geared towards creating user interface elements and so is pixel-based rather than geometric so that a programmer can know precisely which pixels will be touched by a drawing operation. It supports pixel-plotting, lines of any thickness, text, rectangles, rounded rectangles, ovals, arbitrary regions and the drawing of masked-pixmaps. ZDC_ZooLib is an implementation of this API using no OS facilities at all, and thus can be used in server applications without the difficulties that normally poses (gaining access to a window or graphics server from a low-privilege process).

The implementation uses operations on ZBigRegion instances to decide which pixels to touch, decomposes the regions into rectangles and then calls ZDCPixmapBlit to actually do the work.

IFF and QuickTime File Formats

IFF is a venerable data meta-format, introduced in 1985 by Electronic Arts as a standard framing mechanism for multimedia data. In short it defines a nested chunked format, where the file as a whole is considered to be a chunk. A chunk has as its first four bytes a tag that indicates the type of data in the payload, a 32 bit (four bytes again) count of the number of bytes in the payload, and then that number bytes being the payload itself. Some chunk types are known to contain zero or more other chunks in their payload, so an arbitrarily complex hiearchy can be established. There are other details, but chunk types and sizes are IFF's essentials. QuickTime's file format is almost the same, except that the size comes first, and includes the eight bytes of the chunk header.

IFF/QT are simpler than most file formats, but can still be fiddly to work with. To make it easier for me to parse and generate QT files manually (on platforms which don't have QT libraries) I put together a suite of ZooLib streams that take care of all the bookkeeping.

Tuples Defined Rigorously

For most of my career I've been very suspicious of dynamic data representations. After all, what's the point of having a compiler if it isn't provided with enough information about the shape of the data being manipulated to tell you when your code is going wrong. However, that really only works for data created and consumed within a single application. In the mid-90s, every C++ framework worth its salt had a huge bunch of code dedicated to turning arbitrary C++ objects into something that could be serialized and regenerated as objects later; and in fact that's about as far as most people got with CORBA before giving up.

But most objects in a C++ program, if they're objects at all, simply don't need to be serialized. The ones that do have disparate needs, and special cases abound. The approach I've found most flexible and least intrusive is to provide ZTuple. In this context a tuple is isomorphic to a LISP a-list, Python or Cocoa dictionary, Perl or Ruby Hash or to a Java Map. It's simply a list of name/value pairs, where values can be primitives (strings, numbers, raw bytes etc), tuples or lists of values.

ZooLib provides a suite of facilities that read and write tuples to binary streams, generate and parse a well-defined and easy to read text format, and that can read and write appropriate data formats as tuples.

Project Archives