Implementation

This is a very brief overview of the architecture. More detailed information will follow soon.

Main components

Network handler

Handles the lower level communication, at the socket level (TCP-based). It handles the setting up and tearing down of connections, and the receiving and sending of the buffers produced by the upper layers, including incoming buffering.

Message handler

Handles the picking up of buffers queued by the network handler and routes the buffers to the appropriate interpretation context (or creates a new one when necessary).

Encoding/Decoding

Requests (aka queries) and responses are potentially segmented into multiple buffers. The encoding and decoding layer takes care of the segmentation and reassembly. The complexity of this layer is due to the fact that data objects are sent and reassembled in pieces without an additional copy. A large time-series for example might be transmitted over a long series of buffers and therefore state information must be maintainted. The interpretation context is where this state is maintained.

Interpretation context

The interpretation context handles the states of outgoing and incoming requests. It stores the decoding status of the encoder, as well as the state and control stucture of the ongoing interpretation. It also takes care of invoking the interpreter.

Lexing/Parsing/A-Normalization

The lexing is done by flex generated code. In order to use LALR with bison, the lexing stage uses lookahead in some cases in order to determine the correct token.

The parsing is then fairly straightforward and bison is used to generate the code. This stage produces an Abstract Syntax Tree (AST).

Next, in order to simplify the interpreter, the AST is transformed to A-Normal Form (ANF). See for example http://matt.might.net/articles/a-normalization.

Interpreter

The interpreter and environment code are tighly coupled to the copy on write pointers (cow_ptr). The cow_ptr has semantics beyond that of a usual copy on write pointers. In particular it can be used to signify that the pointed object cannot be modified or should be modified in place (reference).

A significant amount of code implements the built-in functions that form the base of the ztsdb API.

Threads

The network handler runs in one thread and the rest (message handler, intepretation contexts, interpreter) runs in another thread. Most likely a future development will assign the sending of temporary values to a separate thread.