From dbd15b27c0535c3adf6d8bd078472373b8d0c064 Mon Sep 17 00:00:00 2001 From: David Cassany Date: Tue, 3 Dec 2024 16:26:59 +0100 Subject: [PATCH 1/3] Add some docs in READMEs Signed-off-by: David Cassany --- README.md | 32 +++++++++++++++++++++++++++++- updatesparser/README.md | 43 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 updatesparser/README.md diff --git a/README.md b/README.md index a622f83..31491fa 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,7 @@ This repository automates and facilitates the creation of the officially distrib 1. Be the single source of truth for all channel .json files that need to be published by the Elemental team 1. Automatically refresh the channels watching container registries (Daily) +1. Automatically generate a tarball per channel including a .packages file per image listing all the packages that are part of the image 1. (Optional) Publish images on GitHub, for development or testing ## Repository Watches config @@ -30,12 +31,17 @@ watches: isoRepo: registry.suse.com/suse/sle-micro-iso/my-flavor-5.5 # How many images to limit per (minor) version. limit: 3 + # The minimum version to start listing the channel. Usefull to skip any + # pre GM build that eventually got released to the registry during release preparation. + minVersion: 2.2.0 ``` ## Usage It is possible, at any moment, to run the `.refresh_channels.sh` script and integrate the changes, if any. -A GitHub [workflow](.github/workflows/refresh-channels.yaml) does it automatically every night, and optionally it can be triggered at any time. +A GitHub [workflow](.github/workflows/refresh-channels.yaml) does it automatically every night, and optionally it can be triggered at any time. + +`.refresh_channels.sh` script will recreate the .json file of a channel and the associated .packages.tar file. Manually crated `.json` files can be directly created and maintained in the `./channels` directory, for example to maintain a channel containing arbitrary images, for development or testing. When this is the case, be mindful of not creating collision with the automated [config.yaml](./config.yaml), otherwise files with the same name will be overwritten. @@ -45,3 +51,27 @@ When this is the case, be mindful of not creating collision with the automated [ The `unstable` or `development` channel is a special static one that is meant to include latest build of the development OBS projects. Hence this is an static JSON only including the ISO and OS container for the current build in projects under or branched from `isv:Rancher:Elemental` in OBS. This channel is not automated or generated by any kind of continuous integration. + +## Elemental channel images + +The Elemental channel images are defined under `.obs/dockerfile/elemental-channel-image`. +This folder is actually used to feed [rancher/elemental-obs](https://github.com/rancher/elemental-obs) +repository which is includes the OBS sources for all Elemental components. + +## Changelog generation for images listed in channels + +1. Changelogs are precomputed and generated at build time, as part of the Dockerfile, only if the `updatesinfo.xml` +files are present. +1. The updatesinfo.xml file is parsed and computed using the [updatesparser](https://github.com/rancher/elemental-channels/blob/main/updatesparser/README.md) utility included in this git repository. +1. There is a changelog file in JSON format per image. +1. Each change log file includes the updates up to its creation date and after the creation date of the previous image. Oldest image in the list includes all the updates up to its creation date. +1. Changelogs are included within the channel image in `/changelogs` folder. Each file is named with the name of the imge it relates to (aka the ManagedOSVersion name) with the .updates.log extension. +1. Within the image sources there is the needed script to checkout the updateinfo.xml files at OBS commit time and the script to eventually iterate over the images listed in the channel .json and precompute the changelog per image at build time. + +## Channels update sequence + +The semi automated process to generate and update channels including chanegelogs is essentially the following: + +1. Within the current repository the `.refresh_channels.sh` script is used in GHA or manually to recreate the channel .json files and the \*.packages.tar files. GHA creates automatic PRs, merging them is a manual process. +2. In [rancher/elemental-obs](https://github.com/rancher/elemental-obs) GHA there is job to keep track of latest changes in this repository and update `elemental-channel-image` OBS package accordingly. Creates automatic PRs, merging them is a manual process. +3. To include updatesinfo.xml files in OBS a manual process of checking out the sources and calling the `update.sh` script with the appropriate product URL is required. This will download the updatesinfo.xml files which can then be committed as part of the sources. diff --git a/updatesparser/README.md b/updatesparser/README.md new file mode 100644 index 0000000..b1440ff --- /dev/null +++ b/updatesparser/README.md @@ -0,0 +1,43 @@ +# Updatesinfo.xml file parsing tool + +This little utility can be used to parse and filter updatesinfo.xml files +to generate changelogs based on packages match, dates and update type. + +By default produces a human readable text output or a JSON following the +updatesinfo update structure. A custom template can also be provided to +fully customize the output. + +## Custom templates + +Any custom templates requires defining four different named golang templates: + +* `header`: This is only included once at the very begining of the output +* `body`: This is the representation of a single update entity in updatesinfo.xml files +* `join`: This is only included in between two updates (aka the separator character or lines between two updates) +* `footer`: This is only included once at the very end of the output + +## Compile + +``` +make build +``` + +## Usage + +``` +$ updatesparser --help +A simple CLI to parser updateinfo XML files + +Usage: + updatesparser [flags] updateinfo + +Flags: + -a, --afterDate string Filter updates released after the given date. Date as a unix timestamp + -b, --beforeDate string Filter updates released before the given date. Date as a unix timestamp + -h, --help help for updatesparser + -j, --json Output in json format + -o, --output string Output file. Defaults to 'stdout' + -p, --packages string Package file list to filter updates modiying any of listed packages + -s, --security Match only security updates + -t, --template string Provides a custom update template file +``` From a9f9d151bb07103ecfcb7a109090b6246d1d9d86 Mon Sep 17 00:00:00 2001 From: David Cassany Date: Tue, 3 Dec 2024 17:00:36 +0100 Subject: [PATCH 2/3] A couple of fixes and rephrases Signed-off-by: David Cassany --- README.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 31491fa..c74e29f 100644 --- a/README.md +++ b/README.md @@ -56,17 +56,16 @@ any kind of continuous integration. The Elemental channel images are defined under `.obs/dockerfile/elemental-channel-image`. This folder is actually used to feed [rancher/elemental-obs](https://github.com/rancher/elemental-obs) -repository which is includes the OBS sources for all Elemental components. +repository which includes the OBS sources for all Elemental components. ## Changelog generation for images listed in channels -1. Changelogs are precomputed and generated at build time, as part of the Dockerfile, only if the `updatesinfo.xml` -files are present. +1. Changelogs are precomputed and generated at build time, as part of the Dockerfile, only if the `updatesinfo.xml` files are present. 1. The updatesinfo.xml file is parsed and computed using the [updatesparser](https://github.com/rancher/elemental-channels/blob/main/updatesparser/README.md) utility included in this git repository. -1. There is a changelog file in JSON format per image. +1. There is a single changelog file in JSON format per image listed within the channel. 1. Each change log file includes the updates up to its creation date and after the creation date of the previous image. Oldest image in the list includes all the updates up to its creation date. -1. Changelogs are included within the channel image in `/changelogs` folder. Each file is named with the name of the imge it relates to (aka the ManagedOSVersion name) with the .updates.log extension. -1. Within the image sources there is the needed script to checkout the updateinfo.xml files at OBS commit time and the script to eventually iterate over the images listed in the channel .json and precompute the changelog per image at build time. +1. Changelogs are included within the channel image in `/changelogs` folder. Each file is named with the name of the imge it relates to (aka the ManagedOSVersion name) with the `.updates.log` extension. +1. Within the image sources there is the needed script to checkout the updateinfo.xml files at OBS commit time and the script to eventually iterate over the images listed in channels and precompute the changelog per image at build time. ## Channels update sequence @@ -74,4 +73,4 @@ The semi automated process to generate and update channels including chanegelogs 1. Within the current repository the `.refresh_channels.sh` script is used in GHA or manually to recreate the channel .json files and the \*.packages.tar files. GHA creates automatic PRs, merging them is a manual process. 2. In [rancher/elemental-obs](https://github.com/rancher/elemental-obs) GHA there is job to keep track of latest changes in this repository and update `elemental-channel-image` OBS package accordingly. Creates automatic PRs, merging them is a manual process. -3. To include updatesinfo.xml files in OBS a manual process of checking out the sources and calling the `update.sh` script with the appropriate product URL is required. This will download the updatesinfo.xml files which can then be committed as part of the sources. +3. To include updatesinfo.xml files in OBS a manual process is required. Call the `update.sh` script on a locak checkout with the appropriate product URL, this will download the updatesinfo.xml files which can then be committed as part of the sources. From 9ec1b8ccaf15e725980dbb425e903661684ab9ef Mon Sep 17 00:00:00 2001 From: David Cassany Viladomat Date: Wed, 4 Dec 2024 09:39:55 +0100 Subject: [PATCH 3/3] Update README.md Co-authored-by: Andrea Mazzotti --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c74e29f..7fbf751 100644 --- a/README.md +++ b/README.md @@ -73,4 +73,4 @@ The semi automated process to generate and update channels including chanegelogs 1. Within the current repository the `.refresh_channels.sh` script is used in GHA or manually to recreate the channel .json files and the \*.packages.tar files. GHA creates automatic PRs, merging them is a manual process. 2. In [rancher/elemental-obs](https://github.com/rancher/elemental-obs) GHA there is job to keep track of latest changes in this repository and update `elemental-channel-image` OBS package accordingly. Creates automatic PRs, merging them is a manual process. -3. To include updatesinfo.xml files in OBS a manual process is required. Call the `update.sh` script on a locak checkout with the appropriate product URL, this will download the updatesinfo.xml files which can then be committed as part of the sources. +3. To include updatesinfo.xml files in OBS a manual process is required. Call the `update.sh` script on a local checkout with the appropriate product URL, this will download the updatesinfo.xml files which can then be committed as part of the sources.