Consistent hashing

In computer science, consistent hashing is a special kind of hashing such that when a hash table is resized, only ${\displaystyle K/n}$ keys need to be remapped on average, where ${\displaystyle K}$ is the number of keys, and ${\displaystyle n}$ is the number of slots.

In contrast, in most traditional hash tables, a change in the number of array slots causes nearly all keys to be remapped because the mapping between the keys and the slots is defined by a modular operation. Consistent hashing is a particular case of rendezvous hashing, which has a conceptually simpler algorithm, and was first described in 1996.[citation needed]. Consistent hashing first appeared in 1997, uses a different algorithm.[1]

History

The term "consistent hashing" was introduced by Karger et al. at MIT for use in distributed caching. This academic paper from 1997 introduced the term "consistent hashing" as a way of distributing requests among a changing population of web servers. Each slot is then represented by a node in a distributed system. The addition (joins) and removal (leaves/failures) of nodes only requires ${\displaystyle K/n}$  items to be re-shuffled when the number of slots/nodes change. The authors mention linear hashing and its ability to handle sequential node addition and removal, while consistent hashing allows buckets to be added and removed in arbitrary order. [1]

Teradata used this technique in their distributed database, released in 1986, although they did not use this term. Teradata still use the concept of a hash table to fulfill exactly this purpose. Akamai Technologies was founded in 1998 by the scientists Daniel Lewin and F. Thomson Leighton (co-authors of the article coining "consistent hashing") to apply this algorithm, which gave birth to the content delivery network industry.

Consistent hashing has also been used to reduce the impact of partial system failures in large web applications to provide robust caching without incurring the system-wide fallout of a failure.[2]

Consistent hashing is also the cornerstone of distributed hash tables (DHTs), which employ hash values to partition a keyspace across a distributed set of nodes, then construct an overlay network of connected nodes that provide efficient node retrieval by key.

Rendezvous hashing, designed at the same time as consistent hashing, achieves the same goals using the very different highest random weight (HRW) algorithm.

Motivations

While running collections of caching machines some limitations are experienced. A common way of load balancing ${\displaystyle n}$  cache machines is to put object ${\displaystyle o}$  in cache machine number ${\displaystyle {\text{hash}}(o)\;\left({\text{mod }}n\right)}$ . But this will not work if a cache machine is added or removed because ${\displaystyle n}$  changes and every object is hashed to a new location. This can be disastrous since the originating content servers are flooded with requests from the cache machines. Hence consistent hashing is needed to avoid swamping of servers.

Consistent hashing maps objects to the same cache machine, as far as possible. It means when a cache machine is added, it takes its share of objects from all the other cache machines and when it is removed, its objects are shared among the remaining machines.

The main idea behind the consistent hashing algorithm is to associate each cache with one or more hash value intervals where the interval boundaries are determined by calculating the hash of each cache identifier. (The hash function used to define the intervals does not have to be the same function used to hash the cached values. Only the range of the two functions need match.) If the cache is removed, its interval is taken over by a cache with an adjacent interval while all the remaining caches are unchanged.

Technique

Consistent hashing is based on mapping each object to a point on a circle (or equivalently, mapping each object to a real angle). The system maps each available machine (or other storage bucket) to many pseudo-randomly distributed points on the same circle.

To find where an object should be placed, the system finds the location of that object's key on the circle; then walks around the circle until falling into the first bucket it encounters (or equivalently, the first available bucket with a higher angle). The result is that each bucket contains all the resources located between each one of its points and the previous points that belong to other buckets.

If a bucket becomes unavailable (for example because the computer it resides on is not reachable), then the points it maps to will be removed. Requests for resources that would have mapped to each of those points now map to the next highest points. Since each bucket is associated with many pseudo-randomly distributed points, the resources that were held by that bucket will now map to many different buckets. The items that mapped to the lost bucket must be redistributed among the remaining ones, but values mapping to other buckets will still do so and do not need to be moved.

A similar process occurs when a bucket is added. By adding new bucket points, we make any resources between those and the points corresponding to the next smaller angles map to the new bucket. These resources will no longer be associated with the previous buckets, and any value previously stored there will not be found by the selection method described above.

The portion of the keys associated with each bucket can be altered by altering the number of angles that bucket maps to.

Monotonic keys

If key values will always increase monotonically, an alternative approach using a hash table with monotonic keys may be more suitable.[citation needed]

Complexity

Asymptotic time complexities for ${\displaystyle N}$  nodes (or slots) and ${\displaystyle K}$  keys
Classic hash table Consistent hashing
add a node ${\displaystyle O(K)}$  ${\displaystyle O(K/N+log(N))}$
remove a node ${\displaystyle O(K)}$  ${\displaystyle O(K/N+log(N))}$
add a key ${\displaystyle O(1)}$  ${\displaystyle O(log(N))}$
remove a key ${\displaystyle O(1)}$  ${\displaystyle O(log(N))}$

The ${\displaystyle O(log(N))}$  complexity for consistent hashing comes from the fact that a binary search among nodes angles is required to find the next node on the ring.[citation needed]

Examples

Known examples of consistent hashing use include:

References

1. ^ a b Karger, D.; Lehman, E.; Leighton, T.; Panigrahy, R.; Levine, M.; Lewin, D. (1997). Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web. Proceedings of the Twenty-ninth Annual ACM Symposium on Theory of Computing. ACM Press New York, NY, USA. pp. 654–663. doi:10.1145/258533.258660.
2. ^ Karger, D.; Sherman, A.; Berkheimer, A.; Bogstad, B.; Dhanidina, R.; Iwamoto, K.; Kim, B.; Matkins, L.; Yerushalmi, Y. (1999). "Web Caching with Consistent Hashing". Computer Networks. 31 (11): 1203–1213. doi:10.1016/S1389-1286(99)00055-9. Archived from the original on 2008-07-21. Retrieved 2008-02-05.
3. ^ Holt, Greg (February 2011). "Building a Consistent Hashing Ring". openstack.org. Retrieved 2019-11-17.
4. ^ DeCandia, G.; Hastorun, D.; Jampani, M.; Kakulapati, G.; Lakshman, A.; Pilchin, A.; Sivasubramanian, S.; Vosshall, P.; Vogels, Werner (2007). "Dynamo: Amazon's Highly Available Key-Value Store" (PDF). Proceedings of the 21st ACM Symposium on Operating Systems Principles. 41 (6): 205–220. doi:10.1145/1323293.1294281. Retrieved 2018-06-07.
5. ^ Lakshman, Avinash; Malik, Prashant (2010). "Cassandra: a decentralized structured storage system". ACM SIGOPS Operating Systems Review. 44 (2): 35–40. doi:10.1145/1773912.1773922.
6. ^ "Design -- Voldemort". www.project-voldemort.com/. Archived from the original on 9 February 2015. Retrieved 9 February 2015. Consistent hashing is a technique that avoids these problems, and we use it to compute the location of each key on the cluster.
7. ^ "Akka Routing". akka.io. Retrieved 2019-11-16.
8. ^
9. ^ "GlusterFS Algorithms: Distribution". gluster.org. 2012-03-01. Retrieved 2019-11-16.
10. ^ "Skylable architecture". Archived from the original on 2015-10-29. Retrieved 2014-06-21.
11. ^ Roughgarden, Tim; Valiant, Gregory (2016-03-28). "Modern Algorithmic Toolbox" (PDF). stanford.edu. Retrieved 2019-11-17.
12. ^ Vishnevskiy, Stanislav (2017-07-06). "How Discord Scaled Elixir to 5,000,000 Concurrent Users". Retrieved 2019-11-17.
13. ^ Eisenbud, Daniel E.; Yi, Cheng; Contavalli, Carlo; Smith, Cody; Kononov, Roman; Mann-Hielscher, Eric; Cilingiroglu, Ardas; Cheyney, Bin; Shang, Wentao; Hosein, Jinnah Dylan. "Maglev: A Fast and Reliable Software Network Load Balancer" (PDF). Retrieved 2019-11-17.