Wikipedia

User:PeterJAClark/Draft of large update to Riak

< User:PeterJAClark
Riak
Developer(s)Community
Basho Technologies
Initial releaseAugust 17, 2009; 15 years ago (2009-08-17)
Stable release
3.0.7 / July 16, 2021; 3 years ago (2021-07-16)[1]
Written inErlang
Operating systemLinux, BSD, macOS, Solaris, Raspian
PlatformIA-32, x86-64, AArch32
TypeNoSQL Database, data store, Cloud storage
LicenseApache License 2.0
Websiteriak.com

Riak (pronounced "ree-ack" [2]) is a distributed NoSQL Key-value data store that offers high availability, fault tolerance, operational simplicity, and scalability.[3] Riak moved to an entirely open-source project in August 2017, with many of the licensed Enterprise Edition features being incorporated.[4] Riak implements the principles from Amazon's Dynamo paper[5] with heavy influence from the CAP Theorem. Written in Erlang, Riak has fault tolerant data replication and automatic data distribution across the cluster for performance and resilience.[6]

Riak has a pluggable backend for its core storage, with the default storage backend being Bitcask.[7] LevelDB is also supported, with other options (such as the pure-Erlang Leveled) available depending on the version.

Riak was originally developed by engineers employed by Basho Technologies[8] and maintained by them until 2017 when the rights were sold to bet365[9][10] after Basho went into receivership[11].

Main features

edit
Fault-tolerant availability
Riak replicates key/value stores across a cluster of nodes with a default n_val of three. In the case of node outages due to network partition or hardware failures, data can still be written to a neighboring node beyond the initial three, and read-back due to its "masterless" peer-to-peer architecture.
Queries
Riak provides a REST-ful API through HTTP and Protocol Buffers for basic PUT, GET, POST, and DELETE functions. More complex queries are also possible, including secondary indexes, search (via Apache Solr), and MapReduce. MapReduce has native support for both JavaScript (using the SpiderMonkey runtime) and Erlang.
Predictable latency
Riak distributes data across nodes with hashing and can provide latency profile, even in the case of multiple node failures.
Storage options
Keys/values can be stored in memory, disk, or both.
Multi-Datacenter replication
Multi-Datacenter replication (MDC) provides uni-directional and bi-direction replication of data between Riak clusters, whether locally for resilience or globally for faster regional access. Uni-directional replication is useful for read-only sinks such as backups and Disaster Recovery sites. Bi-directional replication allows for multiple Riak cluster to have eventually consistent data across vast distances. Complex replication scenarios such as chains, hub-and-spoke and mesh networks are possible due to the Cascades feature, which allows replication of data between clusters that are not directly connected. There are two primary modes of operation: fullsync and realtime. Fullsync mode ensures that all data on the source cluster is replicated to the sink cluster. Only the metadata and changes are transferred, making this fast and efficient. Realtime mode sends updates made to a source cluster to the sink cluster in realtime. These modes are designed to work together for best performance All multi-datacenter replication occurs over multiple concurrent TCP connections to maximize performance and network utilization.
Tuneable consistency
Option to choose between eventual and strong consistency for each bucket.

Main Products

edit

All versions of Riak are now entirely open-source and free, and include the extra features that Basho charged license fees for.

Basho operated a freemium model, wherein they provided free versions of Riak in the form of Riak Core, Riak KV, Riak CS and Riak TS but made their money from licensing more advanced features and SLA-based support. The extra features from the Enterprise Editions have since been integrated into the open source version of Riak KV, as of Riak KV release 2.2.6.[12] and Riak CS 2.1.2 [13]

Riak Core and Riak Core Lite

edit

Riak Core

edit

riak_core[14] is the distributed systems framework that underpins Riak, forming the foundation for all Riak versions. It is being maintained as part of Riak.

Riak Core Lite

edit

riak_core_lite[15] is intended for general use as a base for creating distributed systems.

Riak KV (Key-Value)

edit

Riak KV is a distributed NoSQL database designed to deliver maximum data availability by distributing data across multiple servers, meaning that if one client can reach one server, it should be able to read and write data.[16]KV went through a few names in it's lifetime, starting as Riak then Riak DS (for Data Store) and finally Riak KV (for Key-Value).

When Basho Technologies went into receivership in 2017[11] KV development was picked up by the open source community and has continued into 2021, with 2.2.6 released in 2018 being the first community release of KV. This release integrated some features that were originally restricted to Basho's Enterprise versions of Riak[17].

Version 2.9.0 was the first major community release by the open source community, releasing in November 2019[17], with version 3.0.1 following on August 20th 2020[18]. Development has continued since then with the latest release being version 3.0.7[19].

Removed features

edit

The current version of Riak no longer supports some features in the Enterprise edition of Riak, including:

  • SNMP/JMX support

Separated features in Riak KV 3.0+

edit

The following features of Riak KV 2.x have been removed by default from the Riak build. Specific builds including these features are available.

  • Yokozuna

Riak CS (Cloud Storage)

edit

Originally known as Riak Moss[20](Riak Multi-tenant Object Storage System - MOSS) but named as Riak CS (Cloud Storage) when released, Riak CS was first publicly released in January 2012.

Riak CS (Cloud Storage) is object storage software built on top of Riak KV, Riak’s distributed database. Riak CS is designed to provide simple, highly-available, distributed cloud storage at any scale, and can be used to build cloud architectures or as storage infrastructure for heavy-duty applications and services.[21]

Riak CS also includes an application called Stanchion[22] which is used to manage the serialization of requests. This enables Riak CS to manage globally unique entities like users and bucket names. Serialization in this context means that the entire cluster agrees upon a single value for any globally unique entity at any given time; when that value is changed, the new value must be recognized throughout the entire cluster.

Riak CS was briefly rebranded as Riak S2 to make it more obviously compatible with Amazon S3 but the name did not catch on and it reverted to Riak CS.

In 2021 development for Riak CS was resumed with contributions from TI Tokyo.

Riak TS (Time Series)

edit

Riak TS is an extension to Riak KV optimized for time series data, in that:

  • it supports structured data, with table definition (with a CREATE TABLE call) required before data can be written;
  • data slices from contiguous regions in its primary index (“quanta”) are stored on the same partition;
  • CRUD operations are optimized for speed, at the expense of consistency.

A limited subset of SQL commands was implemented in Riak TS. There is no provision for consistency guarantees between tables (no foreign indexes). In SELECT statements, WHERE clause is supported but HAVING is not. ORDER BY was to appear in a version that was never released.

Riak TS existed as a collection of branches (in separate components of Riak KV such as riak_kv, riak_pb, etc) and not as product with a repository of its own. It was developed by a dedicated team consisting of Gordon Guthrie (leader), Andy Till and Andrei Zavada, with occasional contributions from other developers.

Riak TS was conceived, along with Riak Data Platform project, as an attempt to diversify Basho’s product line, an undertaking many insiders regard as misguided and eventually contributing to Basho's demise.

Licensing and support

edit

Riak was originally licensed using a freemium model: open source versions of Riak KV, Riak CS and Riak TS are available, but end users can pay for additional features and support. However, since Basho entered receivership[11] and bet365[10] (purchasers of all IP) made all Riak products fully open source, all the premium features are now available in the open source versions. Since Basho's demise, community[23] ad-hoc and paid support options have arisen.

Language support

edit

Riak has official clients for Ruby, Java, Erlang and Python. There are also numerous community-supported drivers for other programming languages.[24]

Community development

edit

After bet365[10] purchased the Riak IP[9] , the Riak products were made full open source and work to integrate premium features into the open source versions was completed with the 2.2.6 release.

History

edit

Riak was originally written by Andy Gross and others at Basho Technologies[2] to power a web Sales Force Automation application by former engineers and executives from Akamai. There was more interest in the datastore technology than the applications built on it, so the company decided to build a business around Riak itself, gaining adoption throughout the Fortune 100 and becoming a foundation to many of the world's fastest-growing Web-based, mobile and social networking applications, as well as cloud service providers. Releases after graduation include

Riak KV

edit

Riak 1.0 was released September 10, 2011

Riak KV Releases
Version Date Released Changes
1.0 September 10, 2011 Inition 1.0 Release[25]
1.1 February 21, 2012 Added Riaknostic, enhanced error logging and reporting, improved resiliency for large clusters, and a new graphical operations and monitoring interface called Riak Control.
1.4 July 10, 2013 Added counters, secondary indexing improvements, reduced object storage overhead, handoff progress reporting, and enhancements to MDC replication
2.0 September 2, 2014 Added new data types including sets, maps, registers, and flags simplifying application development. Strong consistency by bucket, full-text integration with Apache Solr, Security, and reduced replicas for Secondary sites.
2.1 April 16, 2015 Added an optimization for many write-heavy workloads – “write once” buckets – buckets whose entries are intended to be written exactly once, and never updated or over-written.
2.2 November 17, 2016 added Support for Debian 8 and Ubuntu 16.04, Solr integration improvements.[26]
2.2.6 May 21, 2018 The first community release. Added support for Multi-Datacentre Replication which was not part of open-source Riak before, added a grow-only set data type, improved data distribution over nodes and cleaned up production test issues.[27]
2.9.0 November 20, 2019 Added early support for TicTac Active Anti-Entropy, support for a new Riak specific backend called Leveled.[17]
2.9.1 February 17, 2020 Implements next-gen replication[28], various changes to tombstones and bucket listing.[29]
2.9.7 August 16, 2020 Improved Active Anti-Entropy and improved Riak's overall stability.
2.9.8 December 7, 2020 Improved leveled functions[30]
2.9.9 August 6, 2021 Leveled stability improvements [31]
3.0.1 August 20, 2020 Adds support for OTP 20, 21, 22 but is not backwards compatible with previous OTP versions.[32]
3.0.2 January 5, 2021 Implements backend changes from 2.9.8, adds a range_check in the Tictac AAE based full-sync replication[18]
3.0.6 May 8, 2021 Adds location-awareness to cluster management[33], along with bug fixes from 3.0.3 and 3.0.4.[34]
3.0.7 July 21, 2021 Reverts Riak erlang runtime system in interactive mode, rather than the embedded mode it was changed to previously.[19]

Riak CS

edit

Riak CS was made open source on March 20, 2013[35]

Riak CS Releases
Version Date Released Changes
0.0.3 January 26, 2012 The first public release of Riak CS. Known as Riak MOSS at the time.[36]
0.1.0 February 25, 2012 Bucket-level Access control, user record changes, Stanchion is now required.[37]
1.0.0 April 2, 2012 Fixes some process/socket leaks, adds a fix to prevent deadlock conditions, new subsystem for user access & storage usage calculations.[38]
1.0.1 April 18, 2012 Fixes a bug that caused requests to hang if a node in the cluster was unavailable[39]
1.1.0 August 20, 2012 Updates user creation, configuration options for anonymous users, more user account controls for admins, Garbage collection for deleted objects, improved performance.[40]
1.2.0 October 23, 2012 Early support for Multi-datacenter replication, support for riak_test integration, bug fixes.[41]
1.2.1 January 23, 2016 Add reduce phase for listing bucket contents to provide backpressure when executing the MapReduce job, Use prereduce during storage calculations, fixed incorrect 404 error when attempting to list contents of nonexistent bucket.[42]
1.2.2 November 8, 2012 Full support for MDC replication, fixed process leaks.[43]
1.3.0 March 20, 2013 Support for multi-part file uploads, bucket polices for restricted principles/conditions, range header. More administrative command controls, support for FreeBSD, SmartOS and Solaris Packaging.[44]
1.3.1 April 4, 2013 Bug fixes[45]
1.4.0 August 12, 2013 Early support for Swift API/Keystone Authentication, improved performance, bug fixes.[46]
1.5.0 July 31, 2014 Adds a multibag technical preview, new debug command, streamlines commands to new `riak-cs admin` command, improved garbage collection, updated lager, new API - Multiple objects, warning logs for manifests, siblings etc.[47]
1.5.1 September 10, 2014 Adds Bucket restrictions, adds sleep interval for manifest updates, updates riak-cs-debug, changes to bucket resolution.[48]
1.5.2 October 9, 2014 Improved logging for failures with Riak, Changes to log output for access stats, adds a script for invalid garbage collection manifest repairs.[49]
1.5.3 December 12, 2014 Add read_before_last_manifest_write option, Adds configurable timeouts for CS interactions with Riak[50]
1.5.4 March 13, 2015 Disable backpressure sleep, Fixes an incorrect path rewrite in S3 API[51]
2.0.0 March 27, 2015 Updates Riak CS to work with Riak 2.0.5, Changes gc_max_workers to gc.max_workers and changed default setting, early support for AWS v4 authentication, adds cuttlefish, storage optimisations.[52]
2.1.0 October 14, 2015 Final Basho release - Backwards compatible with KV 2.0.5, 21.1, Adds a large number of new metrics for health monitoring purposes along with storage usage metrics. Replaced commands with riak-cs-admin equivilants. Garbage collection improvements.[53]
2.1.1 October 14, 2015 Compatible with KV 2.1.3, 2.1.4, 2.2.x and 2.9.x
2.1.2 April 9, 2019 First post-basho release.[54]

Riak TS

edit

Riak TS was originally released in October 2015 [55]

Riak TS Releases
Version Date Released Changes
1.2.0 February 23, 2016 Implements Riak_shell to allow SWQL commands & logging in a single shell in Riak TS. Bug fixes, Multi-Datacenter replication and riak search not supported.[56]
1.3.0 May 4, 2016 Open sources Riak TS, adds a HTTP API, additional SQL commands and support for MDC replication for enterprise users[57]
1.3.1 July 5, 2016 Addresses Data loss bug in 1.3.0.[58]
1.4.0 August 24, 2016 Adds new SQL features, Rolling upgrade/downgrade support, Global data expiry (per cluster).[59]
1.5.0 December 20, 2016 Expands SQL implementation, Improves data storage and improved overall performance.[60]
1.5.1 January 24, 2017 Bug fixes from 1.5.0[61]
1.5.2 February 21, 2017 Bug Fixes from 1.5.1[62]

Users

edit

Notable users include AT&T, Comcast,[63] GitHub,[63] Best Buy,[63] UK National Health Services (NHS),[64] The Weather Channel,[65] and Riot Games.[66]

See also

edit
  1. ^ Riak 3.0.7 release notes, 2021-09-08
  2. ^ a b Sheehy, Justin. "Riak 1.0 Release Party". Vimeo.
  3. ^ Harvey, Cynthia (23 May 2014). "60 Open Source Apps You Can Use in the Cloud". Datamation. Retrieved 5 June 2014.
  4. ^ "Riak KV 2.2.6 Release Notes". github.com. April 26, 2018. Retrieved September 10, 2021.
  5. ^ DeCandia, Giuseppe; Hastorun, Deniz; Jampani, Madan; Kakulapati, Gunavardhan; Lakshman, Avinash; Pilchin, Alex; Sivasubramanian, Swaminathan; Vosshall, Peter; Vogels, Werner (October 14–17, 2007). Dynamo: Amazon's Highly Available Key-value Store (PDF). Proceedings of 21st ACM SIGOPS Symposium on Operating Systems Principles (SOSP '07). Stevenson, Washington, USA: ACM. pp. 205–220. doi:10.1145/1294261.1294281. ISBN 978-1-59593-591-5. Retrieved 5 June 2014.
  6. ^ Morgan, Timothy Prickett (7 May 2014). "Eucalyptus Scales Out AWS Cloud Clone". Enterprise Tech. Retrieved 5 June 2014.
  7. ^ "Basho: Bitcask". docs.riak.com. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  8. ^ "Basho Technologies", Wikipedia, 2020-11-19, retrieved 2021-07-15
  9. ^ a b "Riak NoSQL snapped up by Bet365". ComputerWeekly.com. Retrieved 2021-07-15.
  10. ^ a b c "Bet365", Wikipedia, 2021-07-14, retrieved 2021-07-30
  11. ^ a b c Hill, Rebecca. "End of the road for Basho: Court puts biz into receivership". www.theregister.com. Retrieved 2021-07-15.
  12. ^ "basho/riak". GitHub. Retrieved 2021-07-15.
  13. ^ "basho/riak_cs". GitHub. Retrieved 2021-07-24.
  14. ^ "Riak Core".{{cite web}}: CS1 maint: url-status (link)
  15. ^ "Riak Core Lite".{{cite web}}: CS1 maint: url-status (link)
  16. ^ "Riak KV". docs.Riak.com. 15 July 2021.{{cite web}}: CS1 maint: url-status (link)
  17. ^ a b c "basho/riak". GitHub. Retrieved 2021-07-15.
  18. ^ a b "basho/riak". GitHub. Retrieved 2021-07-15.
  19. ^ a b "Riak KV 3.0.7 Release notes". Riak Github. 2021-07-21. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  20. ^ "Riak Moss". Github. 2012-01-26. Retrieved 2021-08-24.{{cite web}}: CS1 maint: url-status (link)
  21. ^ "Riak Cloud Storage". docs.Riak.com. 15 July 2021.{{cite web}}: CS1 maint: url-status (link)
  22. ^ "Stanchion Application". Tiot.jp. Retrieved 2021-08-24.{{cite web}}: CS1 maint: url-status (link)
  23. ^ "Riak Community site". Riak info. 2021-09-14. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  24. ^ "Riak Client Libraries and Community Code". Docs.riak.com. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  25. ^ "Riak 1.0 release". Riak.com. 2011-09-11. Retrieved 2021-08-26.{{cite web}}: CS1 maint: url-status (link)
  26. ^ "Riak KV 2.2.0 Release Notes". Basho. 2016-11-17. Retrieved 2016-12-21.
  27. ^ "Riak KV 2.2.5 Release Notes". Retrieved 23 June 2018.
  28. ^ "basho/riak_kv". GitHub. Retrieved 2021-07-15.
  29. ^ "basho/riak". GitHub. Retrieved 2021-07-15.
  30. ^ "Riak KV 2.9.8 release notes". Github. 2020-12-07. Retrieved 2021-08-19.{{cite web}}: CS1 maint: url-status (link)
  31. ^ "Release notes 2.9.9". 2021-08-06. Retrieved 2021-08-24.{{cite web}}: CS1 maint: url-status (link)
  32. ^ "basho/riak". GitHub. Retrieved 2021-07-15.
  33. ^ "basho/riak_core". GitHub. Retrieved 2021-07-15.
  34. ^ "basho/riak". GitHub. Retrieved 2021-07-15.
  35. ^ "Riak CS is now open source". Riak.com. 2013-03-20. Retrieved 2021-08-26.{{cite web}}: CS1 maint: url-status (link)
  36. ^ "Riak MOSS 0.0.3". Github. 2012-02-26. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  37. ^ "Riak CS 0.1.0 Release notes". Github. 2012-02-25. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  38. ^ "Moss 1.0 Release notes". Github. 2012-04-02. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  39. ^ "Moss 1.0.1 Release notes". Github. 2012-04-18. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  40. ^ "Riak CS 1.1.0 release notes". Riak CS Github. 2012-08-20. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  41. ^ "Riak CS 1.2 Release Notes". Riak CS Github. 2012-10-23. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  42. ^ "Riak CS 1.2.1 Release notes". Riak CS Github. 2012-10-23. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  43. ^ "Riak CS Release notes". Riak CS Github. 2012-11-08. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  44. ^ "Riak CS Release". Riak CS Github. 2013-03-20. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  45. ^ "Riak CS 1.3.1 Release notes". Riak CS Github. 2013-04-04. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  46. ^ "Riak CS 1.4.0 Release". Riak CS Github. 2013-08-12. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  47. ^ "Riak CS Release". Riak CS Github. 2014-07-31. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  48. ^ "Riak CS Release". Riak CS Github. 2014-09-10. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  49. ^ "Riak CS Releases". Riak CS github. 2014-10-09. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  50. ^ "Riak CS Release". Riak CS Github. 2014-12-12. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  51. ^ "Riak CS Releases". Riak CS Github. 2015-03-13. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  52. ^ "Docs.riak.com". Riak CS Release notes. 2015-03-27. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  53. ^ "Riak CS Release notes". Docs.riak.com. 2015-10-14. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  54. ^ "Riak CS github". Riak CS Github. 2019-04-09. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  55. ^ "Basho releases Riak TS (time-series)". InfoQ. 2015-10-27. Retrieved 2021-08-19.{{cite web}}: CS1 maint: url-status (link)
  56. ^ "Riak TS release notes". docs.riak.com. 2016-02-23. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  57. ^ "Riak TS 1.3.0 Release notes". docs.riak.com. 2016-05-04. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  58. ^ "Riak TS Release notes". docs.riak.com. 2016-07-05. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  59. ^ "Riak TS 1.4.0 Release notes". docs.riak.com. 2016-08-24. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  60. ^ "Riak TS 1.5.0 Release notes". docs.riak.com. 2016-12-20. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  61. ^ "Riak TS 1.5.1 Release notes". docs.riak.com. 2017-01-24. Retrieved 2021-09-02.{{cite web}}: CS1 maint: url-status (link)
  62. ^ "Riak TS 1.5.2 Release notes". docs.riak.com. 2017-02-21. Retrieved 2021-09-14.{{cite web}}: CS1 maint: url-status (link)
  63. ^ a b c "Basho Technologies takes aim at more enterprises with upgrades". 21 February 2013. Retrieved 26 March 2015.
  64. ^ Clark, Jack (10 October 2013). "NHS tears out its Oracle Spine in favour of open source". The Register. Retrieved 5 June 2014.
  65. ^ Henschen, Doug (2 June 2014). "Why Big Data Tools Are Here To Stay: InformationWeek Video". InformationWeek. Retrieved 5 June 2014.
  66. ^ Ptaszek, Michal (16 January 2016). "Chat Service Architecture: Persistence". RiotGames. Retrieved 2 February 2016.
Retrieved from "https://en.wikipedia.org/w/index.php?title=User:PeterJAClark/Draft_of_large_update_to_Riak&oldid=1044283896"