LeanXcale is both vectorised and employs SIMD (single instruction multiple data) processing. It supports secondary indexes and has developed its own technology for primary indexes which, the company claims, combines the benefits of both B+ trees and LSM (log-structured merge) trees. As mentioned, there is no explicit sharding (the manager performs auto-sharding transparently to the application) but the product should scale (elastic scaling is provided) to hundreds of nodes and internal benchmarks (TPC-C and TPC-H like, and YCSB) suggest that the product scales out linearly. Workload management capabilities and dynamic load balancing using an algorithm developed by the company, to guarantee uniform resource usage that enables the linear scale-out. Active-active replication is provided to support high availability and there is another company-developed algorithm to support geo-replication.
Fig 02 Scaling ACID properties
While there are several secret sauces behind LeanXcale, perhaps the most interesting is the way that the product scales ACID properties. As noted previously, each of these is addressed individually, as illustrated in Figure 2, but the key point is that LeanXcale processes transactions in parallel across multiple nodes (where traditional approaches have a single node bottleneck) while maintaining a consistent view across those transactions. The main principles behind enabling this are that the commit processes themselves are separated from the visibility of the committed data, that timestamps are pre-assigned to committing transactions (so that serialization order is defined), and the detection and resolution of conflicts prior to commit.
While there are multiple elements supporting this approach, the three most worthy of discussion are the Snapshot Server, Commit Sequencer, the Conflict Managers and the Loggers. The first two of these supports the product’s snapshot isolation, and which keeps track of the most recent transaction that is consistent.
Each Conflict Manager takes care of a subset of keys and there can be as many as you like. In principle, maintaining concurrency control in this way should be much more efficient for batching purposes. For example, when LeanXcale ran (internally) a TPC-C benchmark, it achieved a twenty times performance boost for batching.
Each Logger takes care of a fraction of the log records and scale in the same way that Conflict Managers do. They run in parallel and can be replicated. The upshot of this approach is that LeanXcale supports multiple durability models: replicated memory, 1-safe and n-safe durability.
Finally, it is worth commenting on the way that query processing works. SQL is translated into a query plan that is represented as a tree of algebraic operators, which themselves are represented in the query engine as Java bytecode. At the leaves of the query plan there are Scan Operators that push down to KiVi predicate filtering, aggregation, grouping and sorting capabilities, that is, all algebraic operators below a join are pushed down.
