Automatic sharding with twemproxy

A few techniques of implementing partitioning in Redis were previously presented, such as range partitioning, hash partitioning, presharding, and consistent hashing. This section will introduce a production-ready tool called twemproxy (pronounced "two-em-proxy"), which is also known as nutcracker. It was created by Twitter, released in 2012, and licensed under Apache License Version 2.0.

twemproxy is a fast and lightweight proxy for Redis and memcached protocols that implements sharding with support for multiple hashing modes, including consistent hashing. It also enables pipelining of requests and responses, and maintains persistent server connections to shard your data automatically across multiple servers. It works on Linux, *BSD, and Smart OS (Solaris). twemproxy is a great tool, and it will help us easily scale Redis horizontally. It has been used in production by companies such as Pinterest, Tumblr, Twitter, Vine, Wikimedia, Digg, and Snapchat.

This...