A distributed workflow runner with focus on performance and simplicity.
Xene is a high performance, distributed workflow execution platform. It lets you create custom workflows in the form of a Directed Acyclic Graph, which can then be executed based on a trigger configured with the workflow.
Xene is built with Golang with a focus on performance, resiliency, and simplicity. Xene internally runs in a way similar to Kubernetes by getting triggered on a level and then reconciling the desired state with the current state. Currently, xene supports Badger as the storage engines, which is pluggable and can be replaced with a similar key-value store.
Xene aims to tackle two main problems in any workflow execution system:
- Low resource footprint
- Flexibility and Simplicity
Xene exploits a lot of small enhancement to reduce the network and memory footprint of the executors and APIServer. APIServer acts as the brains of the whole ecosystem by directing the decisions of scheduling workloads and managing coordination between all the moving components. All the communication between APIServer and Agent is encrypted using mTLS and xene doesn't require Agents to communicate with each other reducing network footprint.
Xene derives a lot of its design decisions from Kubernetes, like:
- Like Kubernetes, xene is also level triggered and does the reconciliation of state described by the user with the actual state.
- It also uses a KeyValue store to store all the states and objects. To reduce the memory footprint, xene supports the use of Badger as a key-value store.
- Objects are reconciled using controllers configured to watch changes on keys in the KVStore.
Some of the main features of Xene are:
- Locally running workflow from provided specs.
- Command line tool for xene(xenectl) to interact with apiserver.
- mTLS based authentication between agent and apiserver.
- JWT based authentication for API server.
- Google OAuth based authentication integration.
- Beta UI based on React and Typescript
- Workflow creation and running on top of Docker based container runtime executor on the agent.
- Secrets management for agents.
- Workflow pipelines run live log streaming and status feed.
- Google based OAuth for APIServer with domain filtered authorization.
If you'd like to contribute to this project, refer to the contributing guide.
You can start with setting up Xene on your system and try solving a few bugs listed here: https://github.com/fristonio/xene/issues
Xene is licensed under MIT License.
If you have any queries regarding the project or just want to say hello, feel free to drop a mail at [email protected].