Feature: ESPEasy-now layer

docs/source/Plugin/P000_commands.repl

@@ -300,6 +300,36 @@
 
     ``Password,<new password>``"
     "
+    ProvisionConfig","
+    :red:`Internal`","
+    Fetch ``config.dat`` as configured in the Provisioning configuration (see Settings Archive)
+
+    ``ProvisionConfig``"
+    "
+    ProvisionSecurity","
+    :red:`Internal`","
+    Fetch ``security.dat`` as configured in the Provisioning configuration (see Settings Archive)
+
+    ``ProvisionSecurity``"
+    "
+    ProvisionNotification","
+    :red:`Internal`","
+    Fetch ``notification.dat`` as configured in the Provisioning configuration (see Settings Archive)
+
+    ``ProvisionNotification``"
+    "
+    ProvisionProvision","
+    :red:`Internal`","
+    Fetch ``provisioning.dat`` as configured in the Provisioning configuration (see Settings Archive)
+
+    ``ProvisionProvision``"
+    "
+    ProvisionRules","
+    :red:`Internal`","
+    Fetch ``rulesN.txt`` as configured in the Provisioning configuration (see Settings Archive)
+
+    ``ProvisionRules,1`` to fetch ``rules1.txt`` "
+    "
     Publish","
     :green:`Rules`","
     Send command using MQTT broker service

docs/source/Tools/Tools.rst

@@ -833,14 +833,44 @@ Especially if the node is hard to reach for a proper clean setup.
 .. image:: images/SettingsArchive_download2.png
 
 After downloading the files, a summary is given.
+
 A returned error can be something like 404 (file not available) or 401 (not authorized).
 These are the standard HTTP error codes.
+The error will be ``-1`` if the host is unreachable.
+
+If a file already exists, the new file is downloaded with ``_tmp`` appended to the filename.
+If successful, the original file will be renamed to one with ``_bak`` appended to the filename and then the ``_tmp`` version is renamed to the original filename.
+However, if the ``_bak`` is present, it may fail to rename the original one, so the operation fails.
+The presence of the ``_bak`` file is also some protection to not being able to fetch a new version, unless the "Delete First" option is checked.
 
 If ''config.dat'' or ''security.dat'' was downloaded, it is very important to do a reboot and not try to change (and save) anything on the ESPeasy node.
 The old settings are still active in memory and if something will be saved, only the changed part may be saved.
 This would corrupt the settings file.
 
 
+With only ``USE_SETTINGS_ARCHIVE`` defined during build, the URL and credentials cannot be stored.
+For this the build must be made with ``USE_CUSTOM_PROVISIONING`` defined.
+
+N.B. ``USE_CUSTOM_PROVISIONING`` is added on 2021-05-16.
+
+
+URL with Settings
+^^^^^^^^^^^^^^^^^
+
+This holds the full URL without file name where the files must be fetched from.
+
+Since builds made after 2021-05-16, the URL may also contain system variables.
+This allows for an URL like: ``http://192.168.10.127/%mac%``
+
+System variables will be converted into an URL encoded form, which may end up like this:
+
+* ``http://192.168.10.127/A0%3a20%3aA6%3a14%3a84%3a81/rules4.txt`` MAC address: ``A0:20:A6:14:84:81``
+
+The URL will not be stored, unless the build is made with ``USE_CUSTOM_PROVISIONING`` defined and the option is checked to save the URL. (option only present when ``USE_CUSTOM_PROVISIONING`` defined)
+
+Using system variables may allow for multi stage setup of a node, as you could for example fetch a rule which may set a variable to a new value and thus new files may be fetched from a different URL.
+
+
 Side Effects on cloning
 -----------------------
 
@@ -854,3 +884,47 @@ If the original node is configured to use static IP, the clone will use the same
 This can render both inaccessible.
 
 
+Provisioning
+============
+
+Added: 2021-05-16
+
+When the build is made with ``USE_CUSTOM_PROVISIONING`` defined, this Settings Archive screen does allow for more settings helping deployment and remote administration of ESPEasy nodes.
+
+All Settings on the Settings Archive page can be stored in a file named ``provisioning.dat``.
+This file also can store the factory default settings like the device model to ease deployment of a large number of nodes.
+
+N.B. The ``USE_SETTINGS_ARCHIVE`` define is needed to allow to edit the ``provisioning.dat`` file, but it is not needed to use the provisioning feature.
+
+
+.. image:: images/SettingsArchive_provisioning.png
+
+As can be seen, the URL and credentials can be stored.
+This will be stored in a file named ``provisioning.dat`` 
+Such a file may also be fetched from a server.
+
+The ``provisioning.dat`` file can also be automatically generated when performing a factory reset.
+For this the (custom) build must be prepared via a number of defined defaults.
+See the ``Custom-sample.h`` file for some examples.
+
+
+Allow Fetch by Command
+^^^^^^^^^^^^^^^^^^^^^^
+
+This checkbox allows provisioning via commands.
+These commands are not restricted, so they can also be given via HTTP or MQTT.
+
+However, they can only be executed when:
+
+* Allow Fetch by Command is enabled
+* the file to download is checked
+* URL (+ optional credentials) is stored
+
+The commands are:
+
+
+* ``ProvisionConfig`` Fetch ``config.dat``
+* ``ProvisionSecurity`` Fetch ``security.dat``
+* ``ProvisionNotification`` Fetch ``notification.dat``
+* ``ProvisionProvision`` Fetch ``provisioning.dat``
+* ``ProvisionRules,1`` Fetch ``rules1.txt``

lib/I2Cdevlib/I2Cdev.cpp

@@ -774,8 +774,8 @@ uint16_t I2Cdev::readTimeout = I2CDEV_DEFAULT_READ_TIMEOUT;
     // added by Jeff Rowberg 2013-05-07:
     // Arduino Wire-style "beginTransmission" function
     // (takes 7-bit device address like the Wire method, NOT 8-bit: 0x68, not 0xD0/0xD1)
-    uint8_t Fastwire::beginTransmission(uint8_t device) {
-        uint8_t twst, retry;
+    byte Fastwire::beginTransmission(byte device) {
+        byte twst, retry;
         retry = 2;
         do {
             TWCR = (1 << TWINT) | (1 << TWEN) | (1 << TWSTO) | (1 << TWSTA);
@@ -794,8 +794,8 @@ uint16_t I2Cdev::readTimeout = I2CDEV_DEFAULT_READ_TIMEOUT;
         return 0;
     }
 
-    uint8_t Fastwire::writeBuf(uint8_t device, uint8_t address, uint8_t *data, uint8_t num) {
-        uint8_t twst, retry;
+    byte Fastwire::writeBuf(byte device, byte address, byte *data, byte num) {
+        byte twst, retry;
 
         retry = 2;
         do {
@@ -821,7 +821,7 @@ uint16_t I2Cdev::readTimeout = I2CDEV_DEFAULT_READ_TIMEOUT;
         twst = TWSR & 0xF8;
         if (twst != TW_MT_DATA_ACK) return 6;
 
-        for (uint8_t i = 0; i < num; i++) {
+        for (byte i = 0; i < num; i++) {
             //Serial.print(data[i], HEX);
             //Serial.print(" ");
             TWDR = data[i]; // send data to the previously addressed device
@@ -835,8 +835,8 @@ uint16_t I2Cdev::readTimeout = I2CDEV_DEFAULT_READ_TIMEOUT;
         return 0;
     }
 
-    uint8_t Fastwire::write(uint8_t value) {
-        uint8_t twst;
+    byte Fastwire::write(byte value) {
+        byte twst;
         //Serial.println(value, HEX);
         TWDR = value; // send data
         TWCR = (1 << TWINT) | (1 << TWEN);
@@ -846,8 +846,8 @@ uint16_t I2Cdev::readTimeout = I2CDEV_DEFAULT_READ_TIMEOUT;
         return 0;
     }
 
-    uint8_t Fastwire::readBuf(uint8_t device, uint8_t address, uint8_t *data, uint8_t num) {
-        uint8_t twst, retry;
+    byte Fastwire::readBuf(byte device, byte address, byte *data, byte num) {
+        byte twst, retry;
 
         retry = 2;
         do {
@@ -913,7 +913,7 @@ uint16_t I2Cdev::readTimeout = I2CDEV_DEFAULT_READ_TIMEOUT;
         TWCR = 0;
     }
 
-    uint8_t Fastwire::stop() {
+    byte Fastwire::stop() {
         TWCR = (1 << TWINT) | (1 << TWEN) | (1 << TWSTO);
         if (!waitInt()) return 1;
         return 0;
@@ -1028,7 +1028,7 @@ uint16_t I2Cdev::readTimeout = I2CDEV_DEFAULT_READ_TIMEOUT;
     twi_Write_Vars *ptwv = 0;
     static void (*fNextInterruptFunction)(void) = 0;
 
-    void twi_Finish(uint8_t bRetVal) {
+    void twi_Finish(byte bRetVal) {
         if (ptwv) {
             free(ptwv);
             ptwv = 0;

lib/I2Cdevlib/I2Cdev.h

@@ -189,12 +189,12 @@ class I2Cdev {
 
         public:
             static void setup(int khz, boolean pullup);
-            static uint8_t beginTransmission(uint8_t device);
-            static uint8_t write(uint8_t value);
-            static uint8_t writeBuf(uint8_t device, uint8_t address, uint8_t *data, uint8_t num);
-            static uint8_t readBuf(uint8_t device, uint8_t address, uint8_t *data, uint8_t num);
+            static byte beginTransmission(byte device);
+            static byte write(byte value);
+            static byte writeBuf(byte device, byte address, byte *data, byte num);
+            static byte readBuf(byte device, byte address, byte *data, byte num);
             static void reset();
-            static uint8_t stop();
+            static byte stop();
     };
 #endif
 

lib/WifiEspNow/.travis.yml

@@ -0,0 +1,57 @@
+language: bash
+os:
+  - linux
+env:
+  matrix:
+    - PLATFORM=esp8266 \
+      HARDWARE=esp8266com/esp8266 \
+      COREGIT=https://github.com/esp8266/Arduino.git \
+      CORECOMMIT=192aaa42919dc65e5532ea4b60b002c4e19ce0ec \
+      BOARD='--board esp8266com:esp8266:nodemcuv2  --pref custom_xtal=nodemcuv2_80 --pref custom_eesz=nodemcuv2_4M1M'
+    - PLATFORM=esp32 \
+      HARDWARE=espressif/esp32 \
+      COREGIT=https://github.com/espressif/arduino-esp32.git \
+      CORECOMMIT=7df50a97d17b0953ea01cad0355410a66bd8b8b4 \
+      BOARD='--board espressif:esp32:heltec_wifi_kit_32 --pref custom_FlashFreq=heltec_wifi_kit_32_80'
+
+before_install:
+  - sudo iptables -A INPUT -p udp -m udp --dport 5353 -j DROP # https://github.com/per1234/arduino-ci-script/issues/1
+  - sudo apt-get update; sudo apt-get install xvfb
+  - /sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_1.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :1 -ac -screen 0 1280x1024x16
+  - export DISPLAY=:1.0
+
+install:
+  - mkdir -p $HOME/arduino_ide; cd $HOME/arduino_ide
+  - curl -L http://downloads.arduino.cc/arduino-1.8.9-linux64.tar.xz | tar xJ --strip 1
+  - export PATH="$HOME/arduino_ide:$PATH"
+  - |
+    mkdir -p $HOME/Arduino/hardware/$HARDWARE
+    cd $HOME/Arduino/hardware/$HARDWARE
+    git init
+    git fetch $COREGIT
+    git -c advice.detachedHead=false checkout $CORECOMMIT
+    git submodule update --init
+    cd tools
+    python get.py
+  - mkdir -p $HOME/Arduino/libraries; cd $HOME/Arduino/libraries
+
+before_script:
+  - cp -r $TRAVIS_BUILD_DIR $HOME/Arduino/libraries/WifiEspNow
+  - arduino $BOARD --save-prefs
+  - arduino --get-pref
+
+script:
+  - |
+    NFAIL=0
+    for SKETCH in $(find $TRAVIS_BUILD_DIR -name '*.ino'); do
+      echo
+      echo
+      if [[ -f $(dirname $SKETCH)/skip-$PLATFORM.txt ]]; then
+        echo -e '\033[0;34m'Skipping $SKETCH '\033[0m'
+        continue
+      fi
+      echo -e '\033[0;36m'Building $SKETCH '\033[0m'
+      echo
+      arduino --verify $SKETCH || NFAIL=$((NFAIL+1))
+    done
+    [[ $NFAIL -eq 0 ]]