Wikipedia

Valkey

Valkey is an open-source in-memory storage, used as a distributed, in-memory key–value database, cache and message broker, with optional durability.[8] Because it holds all data in memory and because of its design, Valkey offers low-latency reads and writes, making it particularly suitable for use cases that require a cache. Valkey is the successor to Redis, the most popular NoSQL database,[9][10][11] and one of the most popular databases overall.[12] Valkey or its predecessor Redis are used in companies like Twitter,[13][14] Airbnb,[15] Tinder,[16] Yahoo,[17] Adobe,[18] Hulu,[19] Amazon[20] and OpenAI.[21]

Valkey
Original author(s)Salvatore Sanfilippo[1][2]
Developer(s)Linux Foundation[3]
Initial releaseMarch 28, 2024; 31 days ago (2024-03-28)[4]
Stable release
7.2.5[5] Edit this on Wikidata / April 16, 2024; 12 days ago (April 16, 2024)
Repository
Written inC
Operating systemUnix-like[6]
Available inEnglish
TypeData structure store, key–value database
LicenseBSD license[7]
Websitevalkey.io Edit this on Wikidata

Valkey supports different kinds of abstract data structures, such as strings, lists, maps, sets, sorted sets, HyperLogLogs, bitmaps, streams, and spatial indices.

The project was developed and maintained by Salvatore Sanfilippo, starting in 2009.[22] From 2015 until 2020, he led a project core team sponsored by Redis Labs.[23]

In 2018, Redis Ltd., the company managing Redis development, licensed some modules under the proprietary SSPL.[24] In 2024, the company suddenly switched its main Redis code from the BSD license to proprietary licenses.[25] This prompted a large portion of the user and developer community to fork the code under the new name Valkey, retaining the BSD license.[3]

History edit

 
Salvatore Sanfilippo, the original developer of Redis (photo taken in 2015)

The original name Redis meant Remote Dictionary Server.[26][27] The Redis project began when Salvatore Sanfilippo, nicknamed antirez, the original developer of Redis, was trying to improve the scalability of his Italian startup, developing a real-time web log analyzer. After encountering significant problems in scaling some types of workloads using traditional database systems, Sanfilippo began in 2009 to prototype a first proof of concept version of Redis in Tcl.[28] Later Sanfilippo translated that prototype to the C language and implemented the first data type, the list. After a few weeks of using the project internally with success, Sanfilippo decided to open source it, announcing the project on Hacker News. The project began to get traction, particularly among the Ruby community, with GitHub and Instagram being among the first companies adopting it.[29][30]

Sanfilippo was hired by VMware in March, 2010.[31][32][33]

In May, 2013, Redis was sponsored by Pivotal Software (a VMware spin-off).[34]

In June 2015, development became sponsored by Redis Labs.[35]

In August 2018, Redis Labs announced the switch to a proprietary source-available software license, Server Side Public License. Initially the change was said to affect only some optional modules primarily developed by Redis Labs.[24]

In October 2018, Redis 5.0 was released, introducing Redis Stream – a new data structure that allows storage of multiple fields and string values with an automatic, time-based sequence at a single key.[36]

In June 2020, Salvatore Sanfilippo stepped down as Redis' sole maintainer. Sanfilippo was succeeded by Yossi Gottlieb and Oran Agra.[37][38]

In 2024, the Linux Foundation created a fork of Redis under the name of Valkey,[39] allowing community maintainers, contributors, and users to continue working on an open source version of the Redis database. This was in response to the Redis company having switched to fully proprietary software licenses for its core repository.[25]

Differences from other database systems edit

Valkey and Redis popularized the idea of a system that can be considered a store and a cache at the same time. It was designed so that data is always modified and read from the main computer memory, but also stored on disk in a format that is unsuitable for random data access. The formatted data is only reconstructed into memory once the system restarts.

Valkey also provides a data model that is very unusual compared to a relational database management system (RDBMS). User commands do not describe a query to be executed by the database engine but rather specific operations that are performed on given abstract data types. Therefore, data must be stored in a way which is suitable later for fast retrieval. The retrieval is done without help from the database system in form of secondary indexes, aggregations or other common features of traditional RDBMS. The Valkey implementation makes heavy use of the fork system call, to duplicate the process holding the data, so that the parent process continues to serve clients while the child process writes the in-memory data to disk.

Popularity edit

According to monthly DB-Engines rankings, Valkey or its predecessor Redis are often the most popular key–value databases.[9] Redis was ranked the #4 NoSQL database in user satisfaction and market presence based on user reviews,[40] the most popular NoSQL database in containers,[41] and the #4 Data store of 2019 by ranking website stackshare.io.[42] It was voted most loved database in the Stack Overflow Developer Survey in 2017, 2018, 2019, 2020 and 2021.[43]

Supported languages edit

Since version 2.6 of Redis, Valkey featured server-side scripting in the language Lua.[44]

Many programming languages have Redis or Valkey language bindings on the client side, including:[45] ActionScript, C, C++, C#, Chicken, Clojure, Common Lisp, Crystal, D, Dart, Delphi,[46] Elixir, Erlang, Go, Haskell, Haxe, Io, Java, Nim, JavaScript (Node.js), Julia, Lua, Objective-C, OCaml, Perl, PHP, Pure Data, Python, R,[47] Racket, Ruby, Rust, Scala, Smalltalk, Swift, and Tcl. Several client software programs exist in these languages.[45]

Data types edit

Valkey and Redis map keys to types of values. An important difference between these and other structured storage systems is that they support not only strings, but also abstract data types:

  • Lists of strings
  • Sets of strings (collections of non-repeating unsorted elements)
  • Sorted sets of strings (collections of non-repeating elements ordered by a floating-point number called score)
  • Hash tables where keys and values are strings
  • HyperLogLogs used for approximated set cardinality size estimation, available since Redis 2.8.9 in April 2014.[36]
  • Stream of entries with consumer groups, allows you to store multiple fields and string values with an automatic, time-based sequence at a single key, available since Redis 5.0 in October 2018[36]
  • Geospatial data through the implementation of the geohash technique, available since Redis 3.2.[48]

The type of a value determines what operations (called commands) are available for the value. Redis supports high-level, atomic, server-side operations like intersection, union, and difference between sets and sorting of lists, sets and sorted sets.

Some other data types are supported based on the Redis Modules API. Some of them are not licensed under the BSD license used by Valkey:[49]

  • JSON – RedisJSON[50] implements ECMA-404 (the JavaScript Object Notation Data Interchange Standard) as a native data type.[51]
  • Search - A query engine for Redis, providing secondary indexing, full-text search, vector similarity search and aggregations.[52]
  • Time series – RedisTimeSeries[53] implements a time series data structure
  • Bloom filter, Cuckoo filter, Count–min sketch, and Top-K – RedisBloom[54] implements a set of probabilistic data structures for Redis
  • Others [55]

Former implementations include:

  • Graph – RedisGraph[56] implements a queryable property graph

Persistence edit

Valkey and Redis typically hold the whole dataset in memory. Versions up to 2.4 could be configured to use what they refer to as virtual memory[59] in which some of the dataset is stored on disk, but this feature is deprecated. Persistence in Valkey can be achieved through two different methods. First by snapshotting, where the dataset is asynchronously transferred from memory to disk at regular intervals as a binary dump, using the Redis RDB Dump File Format. Alternatively by journaling, where a record of each operation that modifies the dataset is added to an append-only file (AOF) in a background process. Valkey can rewrite the append-only file in the background to avoid an indefinite growth of the journal. Journaling was introduced in version 1.1 and is generally considered the safer approach.

By default, Valkey writes data to a file system at least every 2 seconds, with more or less robust options available if needed. In the case of a complete system failure on default settings, only a few seconds of data would be lost.

Replication edit

Valkey supports master–replica replication. Data from any Valkey server can replicate to any number of replicas. A replica may be a master to another replica. This allows Valkey to implement a single-rooted replication tree. Replicas can be configured to accept writes, permitting intentional and unintentional inconsistency between instances. The publish–subscribe feature is fully implemented, so a client of a replica may subscribe to a channel and receive a full feed of messages published to the master, anywhere up the replication tree. Replication is useful for read (but not write) scalability or data redundancy.[60]

Performance edit

When the durability of data is not needed, the in-memory nature of Valkey allows it to perform well compared to database systems that write every change to disk before considering a transaction committed.[26] Valkey operates as a single process and is single-threaded or double-threaded when it rewrites the AOF (append-only file).[61] Thus, a single Valkey instance cannot use parallel execution of tasks such as stored procedures.

Clustering edit

Valkey introduced clustering in April 2015 with the release of Redis version 3.0.[62] The cluster specification implements a subset of Valkey commands: all single-key commands are available, multi-key operations (commands related to unions and intersections) are restricted to keys belonging to the same node, and commands related to database selection operations are unavailable.[63] A Valkey cluster can scale up to 1,000 nodes, achieve "acceptable" write safety and to continue operations when some nodes fail.[64][65]

Use cases edit

Due to the nature of the database design, typical use cases are session caching, full page cache, message queue applications, leaderboards and counting among others.[66] The publish–subscribe messaging paradigm allows real-time communication between servers.

Amazon Web Services offers a managed Redis service called ElastiCache for Redis, Google offers a managed Redis service called Cloud Memorystore,[67] Microsoft offers Azure Cache for Redis in Azure,[68] and Alibaba offers ApsaraDB for Redis in Alibaba Cloud.[69] Most of these services have announced that they will transition to Valkey.[3]

Users edit

Valkey or Redis are being used in companies like Twitter,[13][14][70] AirBnB,[15] Tinder,[16] Yahoo,[17] Adobe,[18] Hulu,[19] and Amazon.[20]

See also edit

References edit

  1. ^ Bernardi, Stefano (January 4, 2011). "An interview with Salvatore Sanfilippo, creator of Redis, working out of Sicily". EU-Startups. Menlo Media. Retrieved 2024-04-07.
  2. ^ Haber, Itamar (July 15, 2015). "Salvatore Sanfilippo: Welcome to Redis Labs". Redis Labs. Retrieved 2024-04-07.
  3. ^ a b c Bobby Borisov (March 29, 2024). "Valkey: A New Redis Alternative Championed by Tech Giants". Linuxiac. Retrieved 2024-04-06.
  4. ^ "Linux Foundation Launches Open Source Valkey Community". www.linuxfoundation.org. Retrieved 2024-04-09.
  5. ^ "7.2.5". 16 April 2024. Retrieved 16 April 2024.
  6. ^ "Introduction to Redis". redis.io. Retrieved 2024-04-07. Redis is written in ANSI C and works in most POSIX systems like Linux, *BSD, OS X without external dependencies.
  7. ^ "valkey/COPYING". Github. June 23, 2020. Retrieved 2024-04-06.
  8. ^ "Redis". Redis. Retrieved 2023-07-22.
  9. ^ a b "DB-Engines Ranking - popularity ranking of key-value stores". DB-Engines. Retrieved 2024-04-07.
  10. ^ Clark, Lindsay. "Redis becomes the most popular database on AWS as complex cloud application deployments surge". www.theregister.com. Retrieved 2023-07-22.
  11. ^ "Instablinks EP 07: Redis™—The Most Popular In-Memory Database Technology". Instaclustr. Retrieved 2023-07-22.
  12. ^ "DB-Engines Ranking". DB-Engines. Retrieved 2023-07-22.
  13. ^ a b Scaling Redis at Twitter, retrieved 2023-07-22
  14. ^ a b Using Redis at Scale at Twitter - by Rashmi Ramesh of Twitter - RedisConf17 -, retrieved 2023-07-22
  15. ^ a b AWS re:Invent 2018: Airbnb's Journey from Self-Managed Redis to ElastiCache for Redis (DAT319), retrieved 2023-07-22
  16. ^ a b "Building resiliency at scale at Tinder with Amazon ElastiCache | AWS Database Blog". aws.amazon.com. 2020-01-30. Retrieved 2023-07-22.
  17. ^ a b AWS re:Invent 2022 - How Yahoo cost optimizes their in-memory workloads with AWS (DAT321), retrieved 2023-07-22
  18. ^ a b AWS re:Invent 2014 | (SDD402) Amazon ElastiCache Deep Dive, retrieved 2023-07-22
  19. ^ a b "Hulu Case Study". Amazon Web Services, Inc. Retrieved 2023-07-22.
  20. ^ a b "Amazon GameOn Database Migration Case Study – Amazon Web Services (AWS)". Amazon Web Services, Inc. Retrieved 2023-07-22.
  21. ^ "Elevated API Errors". status.openai.com. Retrieved 2023-10-28.
  22. ^ "A conversation with Salvatore Sanfilippo, creator of the open-source database Redis". VentureBeat. 2016-06-20. Retrieved 2021-06-29.
  23. ^ Kepes, Ben (July 15, 2015). "Redis Labs hires the creator of Redis, Salvatore Sanfilippo". Network World. Retrieved 2015-08-30.
  24. ^ a b Claburn, Thomas. "Redis has a license to kill: Open-source database maker takes some code proprietary". www.theregister.com. Retrieved 2024-03-21.
  25. ^ a b "LICENSE.txt". GitHub. 20 March 2024. Retrieved 2024-04-07.
  26. ^ a b "FAQ: Redis". Redis.io. Retrieved 2022-02-12.
  27. ^ "Google Groups". groups.google.com. Retrieved 2022-02-25.
  28. ^ Sanfilippo, Salvatore (April 28, 2017). "Tcl prototype of Redis". GitHub Gist. Retrieved 2018-10-08.
  29. ^ Wanstrath, Chris (November 3, 2009). "Introducing Resque". Blog. Retrieved 2018-10-08.
  30. ^ Krieger, Mike (October 31, 2011). "Storing hundreds of millions of simple key-value pairs in Redis". Instagram Engineering Blog. Retrieved 2018-10-08.
  31. ^ Shapira, Gwen (March 17, 2010). "VMware Hires Redis Key Developer – But Why?". Blog. Retrieved 2016-09-25.
  32. ^ Sanfilippo, Salvatore (March 15, 2010). "VMware: the new Redis home". Blog. Retrieved 2016-09-25.
  33. ^ Collison, Derek (March 15, 2010). "VMware: The Console: VMware hires key developer for Redis". VMware Blog. Archived from the original on 2010-03-22. Retrieved 2016-09-25.
  34. ^ Sanfilippo, Salvatore. "Redis Sponsors". Redis.io. Redis Labs. Retrieved 2019-04-11.
  35. ^ Sanfilippo, Salvatore (July 15, 2015). "Thanks Pivotal, Hello Redis Labs". <antirez>. Retrieved 2019-04-03.
  36. ^ a b c "Redis 5.0 is here!". Redis. 22 October 2018. Retrieved 2024-04-07.
  37. ^ "Database maestro Antirez says arrivederci to Redis: Seems he wants an unstructured life writing code, not a structured one managing software". theregister.com. Retrieved 2024-04-07.
  38. ^ "The end of the Redis adventure -". antirez.com. Retrieved 2020-11-10.
  39. ^ "Linux Foundation Launches Open Source Valkey Community". linuxfoundation.org. Retrieved 2024-04-07.
  40. ^ "Best NoSQL Databases: Fall 2015 Report from G2 Crowd". G2 Crowd. Archived from the original on 2015-08-24. Retrieved 2015-08-25.
  41. ^ "8 Surprising Facts about Real Docker Adoption". Datadog. 13 June 2018. Retrieved 2024-04-07.
  42. ^ "🏆 The Top 50 Developer Tools of 2019". StackShare. Retrieved 2020-07-28.
  43. ^ "Developer Survey Results 2021: Most Loved, Dreaded, and Wanted Databases". Stack Overflow. Stack Exchange. Retrieved 2021-08-23.
  44. ^ "EVAL – Redis". redis.io. Retrieved 2024-04-07.
  45. ^ a b "Redis". redis.io. Retrieved 2024-04-07.
  46. ^ "Danieleteti/Delphiredisclient". GitHub. 17 September 2022. Retrieved 2024-04-07.
  47. ^ Lewis, B. W. (5 July 2015). "rredis: "Redis" Key/Value Database Client". The Comprehensive R Archive Network. Retrieved 2019-04-03.
  48. ^ "Redis 3.2 Release Notes". GitHub. Retrieved 2017-03-10.
  49. ^ "Redis Licensing Overview". Redis. Retrieved 2023-09-30.
  50. ^ "RedisJSON - a JSON data type for Redis". oss.redis.com. Retrieved 2024-04-07.
  51. ^ "RedisJSON - a JSON data type for Redis". redisjson.io. Retrieved 2024-04-07.
  52. ^ RediSearch, RediSearch, 2023-09-30, retrieved 2023-09-30
  53. ^ "RedisTimeSeries - Time-Series data structure for Redis". oss.redis.com. Retrieved 2024-04-07.
  54. ^ "RedisBloom - Probabilistic Datatypes Module for Redis". oss.redis.com. Retrieved 2024-04-07.
  55. ^ "Modules". Redis. Retrieved 2023-09-30.
  56. ^ "RedisGraph - a graph database module for Redis". oss.redis.com. Retrieved 2024-04-07.
  57. ^ Kogan, Lior (2023-07-05). "RedisGraph End-of-Life Announcement". Redis. Retrieved 2023-09-30.
  58. ^ FalkorDB, FalkorDB, 2023-09-29, retrieved 2023-09-30
  59. ^ "Virtual Memory". Redis.io. Retrieved 2019-04-11.
  60. ^ "Google Code Archive - Long-term storage for Google Code Project Hosting". code.google.com. Retrieved 2024-04-07.
  61. ^ "Redis on the Raspberry Pi: adventures in unaligned lands - <antirez>". antirez.com. Retrieved 2024-04-07.
  62. ^ "Redis 3.0 Release Notes". GitHub. Retrieved 2017-03-10.
  63. ^ "Cluster Spec". redis.io. Retrieved 2017-03-10.
  64. ^ "Cluster Spec". redis.io. Retrieved 2017-03-10.
  65. ^ "Cluster Tutorial". redis.io. Retrieved 2017-03-10.
  66. ^ "Top 5 Redis use cases - ObjectRocket". ObjectRocket. Rackspace. 7 November 2017. Retrieved 2024-04-07.
  67. ^ "Memorystore: in-memory data store". Google Cloud. Retrieved 2023-02-03.
  68. ^ "Azure Redis Cache - Redis cache cloud service - Microsoft Azure". azure.microsoft.com. Retrieved 2024-04-07.
  69. ^ "ApsaraDB for Redis: A Key Value Database Service - Alibaba Cloud". www.alibabacloud.com. Retrieved 2024-04-07.
  70. ^ "How Twitter Uses Redis to Scale - 105TB RAM, 39MM QPS, 10,000+ Instances - High Scalability". Highscalability.com. Retrieved 2024-04-07.

Further reading edit

External links edit

Retrieved from "https://en.wikipedia.org/w/index.php?title=Valkey&oldid=1220930692"