Description:

This is a talk about the Tokyo Cabinet suite of products and their use in a production system. I will cover the basics of the Tokyo Cabinet suite of products and then dive into the story of into Collective Intellect uses Tokyo Tyrant and Varnish in a production system to manage 1.3 TiB of xml documents. These 1.8 Billion (and growing) documents are stored in a cluster of Tokyo Tyrant hash databases, fronted by a Varnish server, and configured with ruby programs. Areas I will cover are: why we are using Tyrant, our backup and recovery process, and our partitioning and scaling strategy.