Blazing fast Rust, a modern consensus protocol, and a reliable async runtime — this project intends to provide a consensus backbone for the next generation of distributed data storage systems (SQL, NoSQL, KV, Streaming, Graph ... or maybe something more exotic).
The guide is the best place to get started, followed by the docs for more in-depth details.
This crate differs from other Raft implementations in that:
- It is fully reactive and embraces the async ecosystem. It is driven by actual Raft events taking place in the system as opposed to being driven by a
tickoperation. Batching of messages during replication is still used whenever possible for maximum throughput. - Storage and network integration is well defined via two traits
RaftStorage&RaftNetwork. This provides applications maximum flexibility in being able to choose their storage and networking mediums. See the storage & network chapters of the guide for more details. - All interaction with the Raft node is well defined via a single public
Rafttype, which is used to spawn the Raft async task, and to interact with that task. The API for this system is clear and concise. See the raft chapter in the guide. - Log replication is fully pipelined and batched for optimal performance. Log replication also uses a congestion control mechanism to help keep nodes up-to-date as efficiently as possible.
- It fully supports dynamic cluster membership changes according to the Raft spec. See the
dynamic membershipchapter in the guide. With full support for leader stepdown, and non-voter syncing. - Details on initial cluster formation, and how to effectively do so from an application's perspective, are discussed in the cluster formation chapter in the guide.
- Automatic log compaction with snapshots, as well as snapshot streaming from the leader node to follower nodes is fully supported and configurable.
- The entire code base is instrumented with tracing. This can be used for standard logging, or for distributed tracing, and the verbosity can be statically configured at compile time to completely remove all instrumentation below the configured level.
This implementation strictly adheres to the Raft spec (pdf warning), and all data models use the same nomenclature found in the spec for better understandability. This implementation of Raft has integration tests covering all aspects of a Raft cluster's lifecycle including: cluster formation, dynamic membership changes, snapshotting, writing data to a live cluster and more.
If you are building an application using this Raft implementation, open an issue and let me know! I would love to add your project's name & logo to a users list in this project.
Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.
async-raft is licensed under the terms of the MIT License or the Apache License 2.0, at your choosing.
NOTE: the appearance of the "section" symbols § throughout this project are references to specific sections of the Raft spec.