diff --git a/docs/configuration.rst b/docs/configuration.rst index 0386fca2..89476043 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -1,24 +1,5 @@ - How to configure Tezos Reward Distributor? -===================================================== - -Baker Configuration: ------------------------- - -Each baker has its own configuration and policy. A payment system should -be flexible enough to cover needs of bakers. The application uses a yaml -file for loading baker specific configurations. - -Configuration tool can be used to create baking configuration file -interactively. Also an example configuration file is present under -examples directory. For more information on configuration details, please -see our wiki `page `_. - -TRD is designed to work as a linux service. It expects use of tezos -signer for encrypted payment accounts. Unencrypted payment accounts can -be used without tezos signer. If a payment account is encrypted and not -configured to be signed by tezos signer, TRD will freeze. For more -information on payment addresses please refer to our wiki `page `_. +========================================== Email Setup ------------------------ @@ -33,3 +14,72 @@ fee.ini file contains details about transaction fees. Currently the fee value specified under DEFAULT domain is used as fee amount. It is in mutez. +Baker Configuration: +-------------------- + +Each baker has its own configuration and policy. A payment system should +be flexible enough to cover the needs of bakers. The application uses a yaml +file for loading baker specific configurations. + +Configuration tool can be used to create baking configuration file +interactively. Also an example configuration file is present under +examples directory. +By default configuration files are kept under ~/pymnt/cfg directory. +Configuration directory can be changed with "-f" configuration option. +Name of a configuration file should be the baker's address +(e.g. tz1boot1pK9h2BVGXdyvfQSv8kd1LQM6H889.yaml). + +You can use configure.py tool to interactively create your configuration +file, with ease. + +Available configuration parameters are: + +**baking_address** + Address of the baker. It must be an implicit account (tz1). + No alias is allowed. + +**owners_map** + A dictionary of PKH and ratio ( decimal in the range [0-1]) + pairs. Each item in this dictionary represents PKH of each + balance owner and his ratio of the amount he owned in the + total baking balance. Implicit or originated addresses are + accepted. It is important that the sum of all ratios equals + to 1. This map is optional if owners do not want to be paid + for baking rewards, in this case, service fee remains in + baking balance. + +**founders_map** + A dictionary of PKH and ratio (decimal in the range [0-1]) + pairs. Each item in this dictionary represents PKH of each + founder and his ratio of the shares coming from service fees. + Implicit or originated addresses are accepted. It is important + that the sum of all ratios equals to 1. This map is optional + if founders do not want to be paid from service fees, in + this case, service fee remains in baking balance. + +**service_fee** + A decimal in range [0-100]. This is evaluated as a percentage + value. If this value is set to 5, 5% of baking rewards are + kept as a service fee. + +**supporters_set** + A set of PKH values. Each PKH represents a supporter of the + baker. Supporters are not charged with a service fee. Founders + and balance owners are natural supporters, they are not + needed to be added. + +**specials_map** + A dictionary of PKH and fee (decimal in the range [0-100] ) + pairs. This dictionary can be used to set special service + fee values for desired delegators. + +**delegator_pays_xfer_fee** + Default value is true. If set to false, the transfer fee for + each payment is paid by the delegate. Otherwise, the transfer + fee is deducted from the delegator reward. + +**payment_address** + This is the address where payments will be done from. A PKH + or alias of implicit or originated account is accepted. For + more information on the payment address configuration please + refer to the next section. diff --git a/docs/contributers.rst b/docs/contributers.rst index efd6ea49..531fc6c1 100644 --- a/docs/contributers.rst +++ b/docs/contributers.rst @@ -1,6 +1,23 @@ For developers ===================================================== -Please refer to contributions guide_ on wiki pages. +If you are interested in improving our project, this is just great! Our community looks forward to your contribution. Please follow this process: -.. _guide : https://github.com/habanoz/tezos-reward-distributor/wiki/How-to-Contribute +1. Start a bug/feature issue on GitHub about your proposed change. Continue discussion until a solution is decided. +2. Fork the repo. Ideally, create a new branch from your fork for your contribution to make it easier to merge your changes back. +3. Make your changes on the branch you hopefully created in Step 2. Be sure that your code passes existing unit tests. Please add unit tests for your work if appropriate. It usually is. +4. Push your changes to your fork/branch in GitHub. Don't push it to your master! If you do it will make it harder to submit new changes later. +5. Submit a pull request to the repo from your commit page on GitHub. + + +Principles on the application evolution: + +1. Discuss the solution with repository owners and other community members. Start the development after everybody agrees on a solution. +2. Prefer the simpler solution. Try to make as fewer changes as possible. +3. Update documentation wherever it is meaningful. +4. Create and run unit tests for the changes. Please do not sends PRs for untested changes. Payment domain requires the utmost responsibility. +5. Follow naming conventions. +6. Avoid code repetition. Make use of Object Oriented design principles when possible. +7. More configuration parameters do not always mean more flexibility. We do not want the application to turn into configuration hell which nobody can use. +8. The idea of community members is very important because this repo is just a sequence of statements if nobody uses and benefit. +9. A change will be accepted if it will benefit the community. Particular requests which only very few people will take advantage will not be accepted to leave the code as simple as possible. diff --git a/docs/index.rst b/docs/index.rst index a0eb8bee..473fc751 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,27 +1,52 @@ Tezos Reward Distributor (Run & Forget) |Build Status| ====================================================== -DISCLAIMER : TEZOS REWARD DISTRIBUTOR IS PROVIDED AS IS. IT IS UNDER -CONSTANT DEVELOPMENT. EVENT THOUGH IT IS WELL TESTED, PLEASE USE WITH -CARE. ALWAYS MAKE A PRE-RUN IN DRY MODE BEFORE MAKING ACTUAL PAYMENTS. -IF YOU WANT TO RUN IN SERVICE MODE DO IT AFTER YOU ARE CONFIDENT WITH -THE APPLICATION. IN SERVICE MODE DO NOT UPDATE OFTEN. +DISCLAIMER : TEZOS REWARD DISTRIBUTOR IS PROVIDED AS IS. +IT IS UNDER CONSTANT DEVELOPMENT. EVENT THOUGH IT IS WELL +TESTED, PLEASE USE IT WITH CARE. ALWAYS MAKE A PRE-RUN IN +DRY MODE BEFORE MAKING ACTUAL PAYMENTS. IF YOU WANT TO +RUN IN SERVICE MODE DO IT AFTER YOU ARE CONFIDENT WITH +THE APPLICATION. IN SERVICE MODE ONLY UPDATE IF NEEDED. What is Tezos Reward Distributor? ------------------------------------------------ -TRD is a software for distributing baking rewards with delegators. This -is not a script but a full scale application which can run in the -background all the time. It can track cycles and make payments. It does -not have to be used as a service, It can also be used interactively. +TRD is a software for distributing staking rewards of +delegators introduced in detail in this Medium article_. +This is not a script but a full scale application which +can continuously run in the background as a Linux service. +It can track cycles and make payments. However it does +not have to be used as a service, but it can also be +used interactively. -TRD supports complex payments, pays in batches, provides two back ends -for calculations: rpc and tzstats. It was developed and tested extensively by -the community. For more information, please check following Medium article_ -and the source code which can be found in the following Github_ repo. +TRD supports complex payments, pays in batches, provides +two back ends for calculations: rpc and tzstats_. +Developed and tested extensively by the community and the +source code which can be found in the following Github_ repo. + +**Important note:** + +The terms_ of tzstats note that a license is needed for +the commercial use of the API! + +> If you wish to use the Data in a manner that is primarily +> intended for or directed towards commercial advantage or +> monetary compensation (such use, “Commercial Use”), +> KIDTSUNAMI requires that you enter into a separate +> commercial license agreement. Entering into a separate +> commercial license allows us to protect KIDTSUNAMI’s +> investment in the Data and to maintain the integrity of +>the Data. +> +> Please contact us at license@kidtsunami.com for more +> information about Commercial Uses of our Data. .. _article : https://medium.com/@huseyinabanox/tezos-reward-distributor-e6588c4d27e7 +.. _tzstats : https://tzstats.com/ + +.. _terms : https://tzstats.com/terms + .. _Github : https://github.com/habanoz/tezos-reward-distributor .. toctree:: @@ -30,7 +55,11 @@ and the source code which can be found in the following Github_ repo. installation configuration + paymentaddress + tezossigner + tezossignerdocker run + linuxservice contributers Funding diff --git a/docs/linuxservice.rst b/docs/linuxservice.rst new file mode 100644 index 00000000..b57eda1c --- /dev/null +++ b/docs/linuxservice.rst @@ -0,0 +1,49 @@ +Run as Linux Service +==================== + +Alternatively, it is possible to add tezos-reward-distributer as a Linux service. It +can run in the background. + +If docker is used, make sure user is in docker group + +:: + + sudo usermod -a -G docker $USER + +In order to set up the service with default configuration arguments, run +the following command: + +:: + + sudo python3 service_add.py + + +**Note:** + +If you do not want to use the default arguments, append any arguments +you wish to change after service_add.py. They will be appended to +main.py call. For example if you want to change configuration directory: + +:: + + sudo python3 service_add.py -f ~/payment/config/ + +It will create a service file and use it to enable the service. +Once enabled use following commands to start/stop the service. + +:: + + sudo systemctl start tezos-reward.service + sudo systemctl stop tezos-reward.service + +In order to see service status: + +:: + + systemctl status tezos-reward.service + +In order to see logs: + +:: + + journalctl --follow --unit=tezos-reward.service diff --git a/docs/paymentaddress.rst b/docs/paymentaddress.rst new file mode 100644 index 00000000..52f1497e --- /dev/null +++ b/docs/paymentaddress.rst @@ -0,0 +1,44 @@ +Payment Address +=============== + +TRD is designed to work as a linux service. It expects +the usage of the Tezos signer for encrypted payment +accounts. Unencrypted payment accounts can be used +without tezos signer. If a payment account is encrypted +and not configured to be signed by tezos signer, +TRD will freeze. + +An address can only be used for payments if it satisfies the +following criteria: + +- Public key of the address must be revealed. See tezos command + line interface on how to run reveal command on tezos client. + If an address is registered as delegate, there is no need to + run reveal command. + + :: + + ./tezos-client reveal key for + +- Secret key of the address must be known. If the payment + address is an implicit address (tz), its secret key must + be imported. If payment address is an originated address + (KT), secret key of the manager address must be imported. + +- If secret key is encrypted, tezos-signer must be used to + sign payments. For instructions on how to use signer, + please refer to the next section. + +- Use of unencrypted secret key is also possible. The + unencrypted secret key can be obtained from tezbox + web wallet. An unencrypted secret key can be imported + to the client using the command: + + :: + + ./tezos-client import secret key unencrypted:edskXXXXX + +For more information about tezos command line interface please +refer to the official Tezos documentation_. + +.. _documentation : http://tezos.gitlab.io/api/cli-commands.html diff --git a/docs/run.rst b/docs/run.rst index 81acb938..d7f7c2f5 100644 --- a/docs/run.rst +++ b/docs/run.rst @@ -1,6 +1,9 @@ How to run Tezos Reward Distributor? ===================================================== +Command Line Usage +------------------------ + For a list of parameters, run: :: @@ -9,35 +12,39 @@ For a list of parameters, run: The most common use case is to run in mainnet and start to make payments from last released rewards or continue making payments from the cycle -last payment is done. +last payment is done: :: python3 src/main.py -For more example commands please see wiki `page `_. - +Make payments for a single cycle: -Linux Service ------------------------- +:: -Alternatively, it is possible to add tezos-reward-distributer as a Linux service. It -can run in the background. + python3 src/main.py -C 42 -M 3 -If docker is used, make sure user is in docker group +Make pending payments and stop: :: - sudo usermod -a -G docker $USER + python3 src/main.py -M 2 -In order to set up the service with default configuration arguments, run -the following command: +Make pending payments beginning from a cycle and stop: :: - sudo python3 service_add.py + python3 src/main.py -C 30 -M 2 -For more information please refer to this wiki `page `_. +Run in dry-run mode in zeronet, make payments from cycle 30 and exit: +:: + + python3 src/main.py -D -N ZERONET -C 30 -M 3 +Run in dry-run mode in mainnet, make payments from cycle 30 onwards, +for calculations use data provided by tezos node rpc interface: + +:: + python3 src/main.py -C 30 -P rpc diff --git a/docs/tezossigner.rst b/docs/tezossigner.rst new file mode 100644 index 00000000..29a482b7 --- /dev/null +++ b/docs/tezossigner.rst @@ -0,0 +1,39 @@ +How to use the Tezos Signer +=========================== + +Tezos signer daemon can be configured to sign the +operations with the secret key of the account. +There are two steps, first import secret key to +the signer, second tell the client that it can use +particular signer to sign the operations. + +1. Configure Signer + Replace "``" with your alias. + Replace "edesk1XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" + with your encryped private key. IP address and port + selection are up to the user. + + :: + + ./tezos-signer import secret key encrypted:edesk1XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + ./tezos-signer launch socket signer -a 127.0.0.1 -p 22000 -W + +2. Configure Client + Replace "``" with your alias. + Replace "``" with your public key hash. + Use the same host port combination from the previous step. + + :: + + ./tezos-client import secret key tcp://127.0.0.1:22000/ -f + +When the client is required to sign an operation, the operation +is sent to the signer. Signer generates a signature and sends +back to the client. Normally encrypted accounts are imported +to the signer. So, it is necessary to provide encryption password +to the signer at launch. Note that signer generates generic +signatures e.g. sigXXXX but not edsigXXXX. + +For instructions on how to configure signer on a docker image +go to the next section. + \ No newline at end of file diff --git a/docs/tezossignerdocker.rst b/docs/tezossignerdocker.rst new file mode 100644 index 00000000..d5d77322 --- /dev/null +++ b/docs/tezossignerdocker.rst @@ -0,0 +1,111 @@ +Tezos Signer with Docker +======================== + +In Tezos *mainnet* it is not safe to keep private keys unencrypted. +Many use HSM (Hardware Security Module) to keep private keys. +Luckily HSMs are not the only option. The Tezos Signer can be +used to sign transactions/blocks. Tezos-signer should be given +encrypted private key so that it can sign operations. For +security reasons tezos-signer does not keep encryption password +anywhere, it must be typed upon tezos-signer launch. As a result, +it is not possible to run tezos-signer as a daemon. + +The Tezos Signer must be run from shell interactively and +encryption password must be typed. Then this shell session +must be kept open otherwise tezos-signer process dies. +However, if baking service is in the cloud then it is not +practical to keep a terminal session open. + +The proposed solution is to use Tezos docker image in host +network mode. Start the signer and type the password. +Terminate the terminal session. Signer will continue to +work and sign operations. + +DISCLAIMER: SAFETY OF THE PROPOSED METHOD IS NOT ASSURED. +USE OF HSM IS ADVISED. IF AN ATTACKER BREACHES THE MACHINE +TEZOS SIGNER RUNNING IT MAY COPY YOU PRIVATE KEY FROM +TEZOS SIGNER MEMORY. IN DEFAULT MODE TEZOS SIGNER MAY BE +USED BY THE ATTACKER TO SIGN ANY TRANSACTIONS INCLUDING +TEZOS TRANSFERS. BEFORE USING THIS METHOD BEWARE OF THE +DANGERS. + +**IMPORTANT:** + +This need to be tested again for the current version of the +Tezos test network. Some commands and filenames may have +changed! + +1- Setup Tezos Docker Image +--------------------------- + +Example command is for *babylonnet*: + +:: + + wget -O babylonnet.sh https://gitlab.com/tezos/tezos/raw/babylonnet/scripts/alphanet.sh + chmod +x babylonnet.sh + ./babylonnet.sh start + ./babylonnet.sh status + +For mainnet, replace first "babylonnet.sh" with "mainnet.sh" +and have a look at the official Tezos docker_ documentation. + +2- Configure Tezos Signer +---------------------------- + +For mainnet replace "babylonnet" with "mainnet". Replace +"myaddressalias" with your alias. Replace +"edesk1XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" with your +encryped private key. Replace "8fa665779b7b" with result +from docker ps. + +:: + + docker run -it tezos/tezos:babylonnet tezos-signer import secret key mykeyalias encrypted:edesk1XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + docker ps -a + docker commit 8fa665779b7b tezos-babylonnet-signer + docker run --net=host -it tezos-babylonnet-signer tezos-signer launch socket signer -a 127.0.0.1 -p 22000 -W + +3- Configure Tezos Client +-------------------------- + +Replace "myaddressalias" with your alias. Replace +"tz1PKH" with your public key hash. It is up to you +to select host IP and port. + +:: + + ./tezos-client import secret key myaddressalias tcp://127.0.0.1:22000/tz1PKH -f + +## 4- Configure baker and endorser + +Replace "myaddressalias" with your alias. Replace "tz1PKH" +with your public key hash. "~/.tezos-node/" is where node +data is stored and may be different in your system. Also, +baker/endorser binary name may change according to currently +active protocol. + +:: + + ./tezos-baker-003-PsddFKi3 -R tcp://127.0.0.1:22000/tz1PKH run with local node ~/.tezos-node/ myaddressalias + ./tezos-endorser-003-PsddFKi3 -R tcp://127.0.0.1:22000/tz1PKH run myaddressalias + +## 5- Test +To make sure that tezos-signer can survive terminal session +death, run tests after terminating the session the signer +is launched. + +You must see tezos-signer process after running following command. + +:: + + ps -ef | grep tezos-signer + +Transfer small amount of money from your account to make sure +that tezos-signer is signing operations. + +:: + + ./tezos-client transfer 1 from myaddressalias to someaddress --fee 0 + +.. _docker : http://tezos.gitlab.io/introduction/howtoget.html?highlight=docker#docker-images diff --git a/readme.md b/readme.md index 67a84b36..fcce3173 100644 --- a/readme.md +++ b/readme.md @@ -1,75 +1,82 @@ -DISCLAIMER : TEZOS REWARD DISTRIBUTOR IS PROVIDED AS IS. IT IS UNDER CONSTANT DEVELOPMENT. EVENT THOUGH IT IS WELL TESTED, PLEASE USE WITH CARE. ALWAYS MAKE A PRE-RUN IN DRY MODE BEFORE MAKING ACTUAL PAYMENTS. IF YOU WANT TO RUN IN SERVICE MODE DO IT AFTER YOU ARE CONFIDENT WITH THE APPLICATION. IN SERVICE MODE DO NOT UPDATE OFTEN. +DISCLAIMER : TEZOS REWARD DISTRIBUTOR IS PROVIDED AS IS. IT IS UNDER CONSTANT DEVELOPMENT. EVENT THOUGH IT IS WELL TESTED, PLEASE USE IT WITH CARE. ALWAYS MAKE A PRE-RUN IN DRY MODE BEFORE MAKING ACTUAL PAYMENTS. IF YOU WANT TO RUN IN SERVICE MODE DO IT AFTER YOU ARE CONFIDENT WITH THE APPLICATION. IN SERVICE MODE ONLY UPDATE IF NEEDED. ## Tezos Reward Distributor : Run & Forget [![Build Status](https://travis-ci.com/habanoz/tezos-reward-distributor.svg?branch=master)](https://travis-ci.com/habanoz/tezos-reward-distributor) -TRD is a software for distributing baking rewards with delegators. The documentation can be found [here](https://habanoz.github.io/tezos-reward-distributor/). This is not a script but a full scale application which can run in the background all the time. It can track cycles and make payments. It does not have to be used as a service, It can also be used interactively. +TRD is a software for distributing staking rewards of delegators introduced in detail in this [Medium article](https://medium.com/@huseyinabanox/tezos-reward-distributor-e6588c4d27e7). This is not a script but a full scale application which can continuously run in the background as a Linux service. It can track cycles and make payments. However it does not have to be used as a service, but it can also be used interactively. +The documentation can be found [here](https://habanoz.github.io/tezos-reward-distributor/). -TRD supports complex payments, pays in batches, provides two back ends for calculations: rpc and tzstats. Developed and tested extensively by the community. For more information please check following article. +TRD supports complex payments, pays in batches, provides two back ends for calculations: rpc and [tzstats](https://tzstats.com/). Developed and tested extensively by the community. -[Medium article](https://medium.com/@huseyinabanox/tezos-reward-distributor-e6588c4d27e7) +**Important note:** -### Requirements and Setup: +The [terms and conditions](https://tzstats.com/terms) of tzstats note that a license is needed for the commercial use of the API! -Python 3 is required. You can use the following commands to install it. +> If you wish to use the Data in a manner that is primarily intended for or directed towards commercial advantage or monetary compensation (such use, “Commercial Use”), KIDTSUNAMI requires that you enter into a separate commercial license agreement. Entering into a separate commercial license allows us to protect KIDTSUNAMI’s investment in the Data and to maintain the integrity of the Data. +> +> Please contact us at license@kidtsunami.com for more information about Commercial Uses of our Data. -``` +## Requirements and Setup + +Python 3 is required. You can use the following commands to install it. + +```bash sudo apt-get update sudo apt-get -y install python3-pip ``` Download the application repository using git clone: -``` +```bash git clone https://github.com/habanoz/tezos-reward-distributor ``` To install required modules, use pip with requirements.txt provided. -``` +```bash cd tezos-reward-distributor pip3 install -r requirements.txt ``` Regularly check and upgrade to the latest available version: -``` +```bash +git fetch origin #fetches new branches +git status #see the changes git pull ``` -### Sample configuration: -Before running the TRD, you need to configure a few stuff including the baking address and the payout process. -The configuration file should be by default included in `~/pymnt/cfg/`. -``` +## Sample configuration + +Before running the TRD, you need to configure it e.g by adding your staking address and payout process. +The configuration file should be included in the `~/pymnt/cfg/` directory by default. You can use the following command to copy and modify the example configuration: + +```bash cp tezos-reward-distributor/examples/tz1boot1pK9h2BVGXdyvfQSv8kd1LQM6H889.yaml ~/pymnt/cfg/ +nano ~/pymnt/cfg/tz1boot1pK9h2BVGXdyvfQSv8kd1LQM6H889.yaml ``` - -### How to Run: +## How to Run For a list of parameters, run: -``` +```bash python3 src/main.py --help ``` -The most common use case is to run in mainnet and start to make payments from last released rewards or continue making payments from the cycle last payment is done. +The most common use case is to run in **mainnet** and start to make payments for the latest released rewards or continue making payments from the cycle after the last payment was done. -``` +```bash python3 src/main.py ``` -TRD necessitates an interface to get provided with income and delegator data in order to perform the needed calculations. -The default provider is the public rpc node mainnet.tezrpc.me. However, it is possible to change the data provider with the flag -P rpc. +TRD necessitates an interface to get provided with income and delegator data in order to perform the needed calculations. +The default provider is the public rpc node *mainnet.tezrpc.me*. However, it is possible to change the data provider with the flag -P rpc. Please note that in this case, the default node would be localhost:8732. In order to change the node url for the provider, you can give the desired url under the flag -A followed with node_url:port (e.g. -P rpc -A 127.0.0.1:8733). -Please note that the node should be an archive node, and that the port should be the rpc port specified while launching the node. - -For more example commands please see wiki page: - -https://github.com/habanoz/tezos-reward-distributor/wiki/How-to-Run +Please note that the node should be an archive node, and that the port should be the rpc port specified while launching the node. -### Funding +## Funding TRD is an open source project and will stay like this. It is not funded by any organization. A grant request is rejected by Tezos Foundation. However, I will try to continue to enhance the software and support the community.