Gizzard (Scala framework)

Gizzard is an open source sharding framework to create custom fault-tolerant, distributed databases. It was initially used by Twitter and emerged from a wide variety of data storage problems. Gizzard operates as a middleware networking service that runs on the Java Virtual Machine. It manages partitioning data across arbitrary backend datastores, that allows it to be accessed efficiently. The partitioning rules are stored in a forwarding table that maps key ranges to partitions. Each partition manages its own replication through a declarative replication tree. Gizzard handles both physical and logical shards. Physical shards point to a physical database backend whereas logical shards are trees of other shards. In addition Gizzard also supports migrations and gracefully handles failures. The system is made eventually consistent by requiring that all write operations are idempotent and commutative. As operations fail they are retried at a later time. Gizzard is available at GitHub and licensed under the Apache License.