Release 1.4.3

release/RELEASE

@@ -1,3 +1,98 @@
+1.4.3
+This release includes (highlight main items):
+
+IMPORTANT:
+There are changes required to run a new version. Please make sure you do the following:
+Ask the SENSE team to cancel all provisioned services to your Site.
+Modify Configuration Files for a new release. See the 'Changes needed' section.
+Depending on your installation - update the docker image or restart Kubernetes with a new release.
+
+Changes needed:
+1. (Frontend) Port naming changed. Ports in the configuration are now listed without special character changes. Old way:
+```
+  ports:
+    - "Ethernet1/49"
+    - "Ethernet1/51"
+    - "Ethernet1/52"
+  port_Ethernet1-49_capacity: 100000
+  port_Ethernet1-49_desttype: switch
+  port_Ethernet1-49_isAlias: "isAliasLink"
+  port_Ethernet1-49_wanlink: "Ethernet1/49"
+```
+New way:
+```
+  ports:
+    "Ethernet1/49":
+        "capacity": 100000
+        "desttype": switch
+        "isAlias": "isAlias"
+        "wanlink": true
+    "Ethernet1/51": {}
+    "Ethernet1/52": {}
+```
+2. (Agent) Agents will use Bandwidth Parameters from the configuration. Old way:
+```
+  intf_max: 40000
+  max_bandwidth: 40000
+  min_bandwidth: 1000
+```
+New way:
+```
+  bwParams:
+    unit: mbps # Unit type
+    type: guaranteedCapped | bestEffort # Indicates type for Bandwidth Service
+    priority: 0 # Indicates priority compared to other interfaces
+    reservableCapacity: <0%-100%> # (Optional) Reservable capacity for siterm on that interface (Default 100%)
+    minReservableCapacity: <VAL_IN_UNIT> # Indicates minimum Reservable Capacity (1G suggested, no default)
+    maximumCapacity: <VAL_IN_UNIT> # Indicates maximum Capacity of interface
+    granularity: <VAL_IN_UNIT> # Indicates granularity
+    reservableCapacity: <0%-100%> # (Optional) Reservable capacity for all siterm interfaces (Default 100%)
+```
+3. (Frontend) Allows to control reservable Capacity under port/device. This can be set with the parameter `reservableCapacity` either under the device or port. It must be a percentage: <0%-100%> (Default 100%).
+
+
+SiteRM Improvements:
+SiteRM Agent Moved to HELM deployment. See here for details: https://sdn-sense.github.io/Installation.html
+Web UI allows us to see the state of all SiteRM services. Can be seen under https://<url>/servicestates.html
+Web UI allows to refresh configuration, and or delete hosts from SiteRM's database.
+Report NetworkStatus with data plane type status inside the model. This allows Orchestrator/Admins/Users to identify and know dataplane activation status.
+Report NetworkStatus with data-plane type status inside Prometheus output. New dashboards (under each site) deployed here: https://autogole-grafana.nrp-nautilus.io, which allows you to see the status of the data plane over time.
+Report BandwidthService requests inside Prometheus output. New dashboards (under each site) are available here: https://autogole-grafana.nrp-nautilus.io
+Support multiple lifetime scheduling requests for non-overlapping times, but the same VLANs or IP ranges.
+New APIs, which allow to deletion of old Debug calls, hosts, network devices.
+Routing information (received from Network devices) was removed from the model.
+A New Service, DBCleaner added to cleanup database of old entries.
+Any request is checked for conflicts. Checks are done: vlan overlap, ip overlap, bandwidth exceeding availability.
+The Debugger Service was rewritten to use threading (non-blocking execution). This allows to run multiple Debug services at the same time.
+Frontend introduced a new Debug service (still beta) with an idea to allow issue ping/traceroute from network devices for debugging purposes. (Expected in next release)
+Default txqueulen/mtu same as the master interface, if not specified in the agent configuration.
+If the port is not switchport type - it will be ignored by SiteRM and will not be added to the model.
+Allow to control site which VLANs to put inside model. Parameter `allvlans: True|False` indicates if the device should model all VLANs or only SENSE VLANs inside the model; (Default False)
+Docker/Podman startup script updates: check certificates and configuration files for validity.
+Documentation for all SiteRM Frontend and Agent parameters. See https://sdn-sense.github.io/Configuration.html
+
+
+Network Device improvements:
+(ALL) Implement functionwrapper/classwrapper with verbose logging (log timings, function parameters, output).
+(ALL) Separate VLAN and BGP apply in separate calls. Change order for VLAN/BGP Configuration. Order is now: VLAN, PREFIX-LIST, ROUTE-MAP, BGP.
+(SONiC,DellOS9) Enabled soft-reconfiguration for peering.
+(SONiC) Fix byte/str/int - diff versions of SONiC Release Python output returns either bytes/str/int for ASN Number.
+(SONiC) VRF Support.
+(SONiC) Retry any SONiC command in case of failure (up to 3 times). Visible if 2 modifications happen at the same time.
+(Arista EOS) Older Arista EOS Bandwidth reporting fixed.
+(Dell OS 10) VRF Support, BGP Fix, split prefix-list creation.
+(Dell OS9)(Beta feature) Support traffic police, based on service request if `rate_limit: True|False` is set under device port. (Default False)
+
+Supported systems: x86_64 and ppc64le(beta)
+Installation details: https://sdn-sense.github.io/Installation.html
+
+Sites are encouraged to always use the "latest" or "latest-ppc64le" version. If there is a need to enforce a specific version, please see the details below:
+Docker version for this release:
+x86_64:
+  Agent: sdnsense/site-agent-sense:latest-20240429 (Or use latest)
+  Frontend: sdnsense/site-rm-sense:latest-20240429 (Or use latest)
+
+
 1.3.12
 This release includes (highlight main items):
 

release/release_notes

@@ -2,39 +2,92 @@ This release includes (highlight main items):
 
 IMPORTANT:
 There are changes required to run a new version. Please make sure you do the following:
-Ask SENSE team to cancel all provisioned services to your Site;
-Make sure you update your deployment scripts to use new way to define network devices: https://sdn-sense.github.io/NetControlAnsible.html
-Depending on your installation - use to update docker image or restart kubernetes with new configuration.
+Ask the SENSE team to cancel all provisioned services to your Site.
+Modify Configuration Files for a new release. See the 'Changes needed' section.
+Depending on your installation - update the docker image or restart Kubernetes with a new release.
+
+Changes needed:
+1. (Frontend) Port naming changed. Ports in the configuration are now listed without special character changes. Old way:
+```
+  ports:
+    - "Ethernet1/49"
+    - "Ethernet1/51"
+    - "Ethernet1/52"
+  port_Ethernet1-49_capacity: 100000
+  port_Ethernet1-49_desttype: switch
+  port_Ethernet1-49_isAlias: "isAliasLink"
+  port_Ethernet1-49_wanlink: "Ethernet1/49"
+```
+New way:
+```
+  ports:
+    "Ethernet1/49":
+        "capacity": 100000
+        "desttype": switch
+        "isAlias": "isAlias"
+        "wanlink": true
+    "Ethernet1/51": {}
+    "Ethernet1/52": {}
+```
+2. (Agent) Agents will use Bandwidth Parameters from the configuration. Old way:
+```
+  intf_max: 40000
+  max_bandwidth: 40000
+  min_bandwidth: 1000
+```
+New way:
+```
+  bwParams:
+    unit: mbps # Unit type
+    type: guaranteedCapped | bestEffort # Indicates type for Bandwidth Service
+    priority: 0 # Indicates priority compared to other interfaces
+    reservableCapacity: <0%-100%> # (Optional) Reservable capacity for siterm on that interface (Default 100%)
+    minReservableCapacity: <VAL_IN_UNIT> # Indicates minimum Reservable Capacity (1G suggested, no default)
+    maximumCapacity: <VAL_IN_UNIT> # Indicates maximum Capacity of interface
+    granularity: <VAL_IN_UNIT> # Indicates granularity
+    reservableCapacity: <0%-100%> # (Optional) Reservable capacity for all siterm interfaces (Default 100%)
+```
+3. (Frontend) Allows to control reservable Capacity under port/device. This can be set with the parameter `reservableCapacity` either under the device or port. It must be a percentage: <0%-100%> (Default 100%).
 
 
 SiteRM Improvements:
-Support VirtualPorts (Fake Ports and is Alias)
-Improved SNMP Monitoring and Mac information parsing
-Support QoS allowOvercommit flag. Allow site’s
-to control overcommit or not for L3 QoS.
-Delta Timing and Overlap check imporvements;
-Report Network Status of any configuration (L2,L3,Server Vlan, IP, MTU, QoS). Reporting done inside the model and in Prometheus output.
-Report QoS Requests in a Prometheus output.
-Allow sites to control allvlans flag - either report all vlans from network/host devices or only controlled ones.
-WebUI improvements (delta show, topology map)
-Separation of Ansible-templates and simplification for sites to specific controlled devices. Please look documentation here https://sdn-sense.github.io/NetControlAnsible.html - for a new way to configure control of network devices.
-Before start docker container (either FE or Agent) check all Certificates, configuration files and make sure they are valid.
-FE and Agent use docker volume.
-Upgrade osg-ca-certs (Needed to support Incommon V3)
+SiteRM Agent Moved to HELM deployment. See here for details: https://sdn-sense.github.io/Installation.html
+Web UI allows us to see the state of all SiteRM services. Can be seen under https://<url>/servicestates.html
+Web UI allows to refresh configuration, and or delete hosts from SiteRM's database.
+Report NetworkStatus with data plane type status inside the model. This allows Orchestrator/Admins/Users to identify and know dataplane activation status.
+Report NetworkStatus with data-plane type status inside Prometheus output. New dashboards (under each site) deployed here: https://autogole-grafana.nrp-nautilus.io, which allows you to see the status of the data plane over time.
+Report BandwidthService requests inside Prometheus output. New dashboards (under each site) are available here: https://autogole-grafana.nrp-nautilus.io
+Support multiple lifetime scheduling requests for non-overlapping times, but the same VLANs or IP ranges.
+New APIs, which allow to deletion of old Debug calls, hosts, network devices.
+Routing information (received from Network devices) was removed from the model.
+A New Service, DBCleaner added to cleanup database of old entries.
+Any request is checked for conflicts. Checks are done: vlan overlap, ip overlap, bandwidth exceeding availability.
+The Debugger Service was rewritten to use threading (non-blocking execution). This allows to run multiple Debug services at the same time.
+Frontend introduced a new Debug service (still beta) with an idea to allow issue ping/traceroute from network devices for debugging purposes. (Expected in next release)
+Default txqueulen/mtu same as the master interface, if not specified in the agent configuration.
+If the port is not switchport type - it will be ignored by SiteRM and will not be added to the model.
+Allow to control site which VLANs to put inside model. Parameter `allvlans: True|False` indicates if the device should model all VLANs or only SENSE VLANs inside the model; (Default False)
+Docker/Podman startup script updates: check certificates and configuration files for validity.
+Documentation for all SiteRM Frontend and Agent parameters. See https://sdn-sense.github.io/Configuration.html
 
-Network Device improvements:
-New NOS Support: Cisco Nexus 9 (Support Vlan creation/deletion, IPv4/6 assignment, BGP Configuration)
-SONiC: Use `set ipv6 next-hop prefer-global` and not add link-local address in the routing table. This issue is visible only between Cisco and SONiC BGP Peering.
-AristaEOS: Catch errors and report back to SiteRM.
 
-Code improvement:
-Any new code or modified code - requires to run linter.sh This includes for Python: black, isort, pylint. For yaml - yamllint, For bash - bashlint. 
+Network Device improvements:
+(ALL) Implement functionwrapper/classwrapper with verbose logging (log timings, function parameters, output).
+(ALL) Separate VLAN and BGP apply in separate calls. Change order for VLAN/BGP Configuration. Order is now: VLAN, PREFIX-LIST, ROUTE-MAP, BGP.
+(ALL) New feature to allow split any commands on device (if needed). This is controlled by ansible templates (used for now only by Dell OS 10)
+(SONiC,DellOS9) Enabled soft-reconfiguration for peering.
+(SONiC) Fix byte/str/int - diff versions of SONiC Release Python output returns either bytes/str/int for ASN Number.
+(SONiC) VRF Support.
+(SONiC) Retry any SONiC command in case of failure (up to 3 times). Visible if 2 modifications happen at the same time.
+(Arista EOS) Older Arista EOS Bandwidth reporting fixed.
+(Dell OS 10) VRF Support, BGP Fix, split prefix-list creation.
+(Dell OS9)(Beta feature) Support traffic police, based on service request if `rate_limit: True|False` is set under device port. (Default False)
 
-Supported systems: x86_64 and ppc64le
+Supported systems: x86_64 and ppc64le(beta)
 Installation details: https://sdn-sense.github.io/Installation.html
 
-Sites are encouraged always use "latest" or "latest-ppc64le" version. If there is need to enforce specific version, please see details below:
+Sites are encouraged to always use the "latest" or "latest-ppc64le" version. If there is a need to enforce a specific version, please see the details below:
 Docker version for this release:
 x86_64:
-  Agent: sdnsense/site-agent-sense:latest-20231207 (Or use latest)
-  Frontend: sdnsense/site-rm-sense:latest-20231207 (Or use latest)
+  Agent: sdnsense/site-agent-sense:latest-20240429 (Or use latest)
+  Frontend: sdnsense/site-rm-sense:latest-20240429 (Or use latest)

release/version-update.sh

@@ -31,18 +31,22 @@ NEW_VERSION=$1
 # Check if this version is not already released on github.
 # If it is - fail
 git_tag_conflict=`git tag | grep $NEW_VERSION`
-if [ $git_tag_conflict -ne 0 ];
+if [[ $git_tag_conflict -ne 0 ]];
   then
     echo "This $NEW_VERSION already exists in git repo. Please make sure you create unique version";
     exit 1;
   fi;
 
-
-sed -i "s/.*VERSION = .*/VERSION = '$NEW_VERSION'/" $FDIR/../setupUtilities.py
-sed -i "s/.*__version__ = .*/__version__ = '$NEW_VERSION'/" $FDIR/../src/python/__init__.py
-sed -i "s/.*__version__ = .*/__version__ = '$NEW_VERSION'/" $FDIR/../src/python/SiteFE/__init__.py
-sed -i "s/.*__version__ = .*/__version__ = '$NEW_VERSION'/" $FDIR/../src/python/SiteRMLibs/__init__.py
-sed -i "s/.*__version__ = .*/__version__ = '$NEW_VERSION'/" $FDIR/../src/python/SiteRMAgent/__init__.py
+sed -i .bak "s/.*VERSION = .*/VERSION = '$NEW_VERSION'/" $FDIR/../setupUtilities.py
+rm -f $FDIR/../setupUtilities.py.bak
+sed -i .bak "s/.*__version__ = .*/__version__ = '$NEW_VERSION'/" $FDIR/../src/python/__init__.py
+rm -f $FDIR/../src/python/__init__.py.bak
+sed -i .bak "s/.*__version__ = .*/__version__ = '$NEW_VERSION'/" $FDIR/../src/python/SiteFE/__init__.py
+rm -f $FDIR/../src/python/SiteFE/__init__.py.bak
+sed -i .bak "s/.*__version__ = .*/__version__ = '$NEW_VERSION'/" $FDIR/../src/python/SiteRMLibs/__init__.py
+rm -f $FDIR/../src/python/SiteRMLibs/__init__.py.bak
+sed -i .bak "s/.*__version__ = .*/__version__ = '$NEW_VERSION'/" $FDIR/../src/python/SiteRMAgent/__init__.py
+rm -f $FDIR/../src/python/SiteRMAgent/__init__.py.bak
 
 echo "IMPORTANT: Please update release_notes file with changes from previous version"
 echo "Issue `git add` command and add all new version files"

setupUtilities.py

@@ -26,7 +26,7 @@
 # src/python/SiteFE/__init__.py
 # src/python/SiteRMAgent/__init__.py
 # src/python/SiteRMLibs/__init__.py
-VERSION = '1.3.12'
+VERSION = '1.4.3'
 
 
 def get_path_to_root(appendLocation=None):

src/python/SiteFE/__init__.py

@@ -9,4 +9,4 @@
 # src/python/SiteFE/__init__.py
 # src/python/SiteRMAgent/__init__.py
 # src/python/SiteRMLibs/__init__.py
-__version__ = '1.3.12'
+__version__ = '1.4.3'

src/python/SiteRMAgent/__init__.py

@@ -6,4 +6,4 @@
 # src/python/SiteFE/__init__.py
 # src/python/SiteRMAgent/__init__.py
 # src/python/SiteRMLibs/__init__.py
-__version__ = '1.3.12'
+__version__ = '1.4.3'

src/python/SiteRMLibs/__init__.py

@@ -7,4 +7,4 @@
 # src/python/SiteFE/__init__.py
 # src/python/SiteRMAgent/__init__.py
 # src/python/SiteRMLibs/__init__.py
-__version__ = '1.3.12'
+__version__ = '1.4.3'

src/python/__init__.py

@@ -5,4 +5,4 @@
 # src/python/SiteFE/__init__.py
 # src/python/SiteRMAgent/__init__.py
 # src/python/SiteRMLibs/__init__.py
-__version__ = '1.3.12'
+__version__ = '1.4.3'