Added minio migration script

[sougata-progress]

story : CHEF-16458

[mwrock]

I don't see where the migration is happening in this PR. It looks like it checks if the latest core/minio is installed and if not it installs it. What it needs to do is check if the core/minio is <= 2023-11-01T01-57-10Z. If core/minio is not installed of if the version is > 2023-11-01T01-57-10Z then do nothing and let the provision.sh script do its thing. If they are <= 2023-11-01T01-57-10Z then their minio data needs to be migrated (see https://github.com/habitat-sh/on-prem-builder/blob/main/scripts/minio-update.sh).

You will want to test this by having a stable stand alone builder server setup with some installed packages. Then follow https://github.com/habitat-sh/on-prem-builder/blob/main/on-prem-docs/upgrading.md to do an upgrade. After the upgrade you should be on the LTS minio and all your packages and package metadata should still be visible in the builder UI and you should be able to download them with the hab binary too.

Note that before doing the update, you should set BLDR_CHANNEL to unstable in the bldr.env file to get the latest builder packages.

[mwrock]

Another thing in addition to the comments, what if any of these is not installed: 'core/aws-cli' 'core/bc' 'core/coreutils'? I think we need to make sure they are installed and install if not present. At least aws-cli. Need to check if we need the others.

[mwrock]

I did some testing on this yesterday and here are some observations I have made:

• After migrating, there are some leftover files and folders in the scripts directory. This includes a file called minio-update-bldr-bucket-objects.txt and a directory named minio-update-bldr-bucket-objects. If another migration is attempted on the same machine, these files could negatively impact that migration.
• I noticed a folder called .minio.sys in the data directory. It has a config/config.json file that has some version metadata. I'm not sure what these version numbers correspond to but it may be worth researching. Perhaps it would be better to look here to determine if a migration is necessary instead of the version of core/minio that is installed.
• I wanted to scratch my migration and "rollback" to the old data. I uninstalled the latest core/minio, ran ./uninstall.sh, and copied the data-bkp data back into data. When I reinstalled the on-prem-stable builder packages, minio was not able to come up with errors that it was on a old version.

[mwrock]

I was looking at this again today. I wanted to look more closely at the .mionio.sys directory that gets created in the data directory. It has a format.json file. On the stable builder-minio, the contents of this file is:

{"version":"1","format":"fs","id":"b90e4a35-9afe-4eeb-8cbb-5d5e892282ad","fs":{"version":"2"}}

On LTS it is:

{"version":"1","format":"xl-single","id":"f9ad35cf-bfa4-43ff-bf48-b5f4440abf0d","xl":{"version":"3","this":"2a3a6280-f406-474a-9103-d81eb7a2a971","sets":[["2a3a6280-f406-474a-9103-d81eb7a2a971"]],"distributionAlgo":"SIPMOD+PARITY"}}

I think that the value of format is a better indicator of when a migration is required than examining the version of core/minio. Basically if it is fs then a migration is required.

That all said, I discovered something else totally by accident. That is if you just update the habitat/builder-minio service from stable to LTS and delete the data/.minio.sys directory between unloading the old and loading the new. The migration magically "just happens." If this is truly the case and I'm not imagining things, we could do away with all of this complex migration stuff and just add a few lines to an init or run hook that looks at the format.json and if it has a format of fs then delete the .minio.sys folder before starting minio.