Release documentation

[robaerd]

This PR adds a first draft of the TODO’s for the automated release docu. Discussed in #3536

I would propose to use a parameterized pipeline for the release. This would give us the option to move the „Increment Version Number“ step into the pipeline and minimize the manual interaction.

Furthermore I would also propose adding an additional stage to the release pipeline which installs the deb packages and runs the tests on the installed version. This would require a docker image without the jenkins user or a docker image where the jenkins user has sudo permissions.

I adde these tasks into the TODO section of the doc.

Basics

These points need to be fulfilled for every PR:

• Short descriptions of your changes are in the release notes
  (added as entry in doc/news/_preparation_next_release.md which
  contains _(my name)_)
  Please always add something to the release notes.
• Details of what you changed are in commit messages
  (first line should have module: short statement syntax)
• References to issues, e.g. close #X, are in the commit messages.
• The buildservers are happy.
• The PR is rebased with current master.

If you have any troubles fulfilling these criteria, please write
about the trouble as comment in the PR. We will help you.
But we cannot accept PRs that do not fulfill the basics.

Checklist

Check relevant points but please do not remove entries.
For docu fixes, spell checking, and similar none of these points below
need to be checked.

• I added unit tests for my code
• I fully described what my PR does in the documentation
  (not in the PR description)
• I fixed all affected documentation
• I added code comments, logging, and assertions as appropriate (see Coding Guidelines)
• I updated all meta data (e.g. README.md of plugins and METADATA.ini)
• I mentioned every code not directly written by me in THIRD-PARTY-LICENSES

Review

Reviewers will usually check the following:

• Documentation is introductory, concise, good to read and describes everything what the PR does
• Examples are well chosen and understandable
• Code is conforming to our Coding Guidelines
• APIs are conforming to our Design Guidelines
• Code is consistent to our Design Decisions

Labels

If you are already Elektra developer:

• Add the "work in progress" label if you do not want the PR to be reviewed yet.
• Add the "ready to merge" label if the basics are fulfilled and you also
  say that everything is ready to be merged.

[markus2330]

Thank you!

[markus2330]

inspect the changes done to the git repo, like what scripts/dev/update-infos-status did.

Maybe you can also add a git diff and git log -p of what happened in the release pipeline.

[robaerd]

Regarding the git diff and git log -p, do you think it would be better to

1. run these commands inside the release pipeline and add the output to the artifacts or
2. keep archiving the git repositories in the dry-run option and run these commands locally?

I would prefer former since the latter option would introduce an overhead.

[markus2330]

I agree we should avoiding archiving the whole git repo. And storing the whole diff is probably also too big and not really needed. Actually, we only need to create the git statistics in the pipeline and add the output to the artifacts.

Copying the statistics into the release notes can be a manual step, as usually you need to modify them a bit (formatting, remove uninteresting data and so on...). As this is always a bit subjective, it is actually good if the raw statistics are somewhere else (in the artifacts), too.

[markus2330]

More convenient probably would be if you actually create the release notes (replace the VERSION, copy the statistics and hash sums there and so on). After this is done, the release author simply modifies (e.g. reformat) what was published from the release pipeline. This would, however, already be a quite advanced convenience-feature and is definitely not a must-have.

[mpranj]

I agree, archiving the git repos was only an intermediate step.

Do you think it makes sense to use a manual step in the pipeline to approve the changes before pushing evreything? (e.g. https://www.jenkins.io/doc/pipeline/steps/pipeline-input-step/)
The goal would be to prevent that an accidental error in the release pipeline is pushed to master. On the other hand it's not very elegant when everything goes right anyway.

[robaerd]

Do you think it makes sense to use a manual step in the pipeline to approve the changes before pushing evreything? (e.g. https://www.jenkins.io/doc/pipeline/steps/pipeline-input-step/)

I think the manual step would be a good alternative to the dry-run option. We could inspect the artifacts, git statistics etc. without running the release pipeline twice. This would also prevent pushing errors to master if something went wrong during the non-dry-run.

[markus2330]

Yes, I also like the idea of the manual input instead of the dry-run. I didn't know it is possible. I think we can give it a try.

[markus2330]

Better to have these TODOs in #3519 (and this one actually already is)

[markus2330]

https://github.com/ElektraInitiative/kconfig and https://github.com/ElektraInitiative/gsettings-tests would be ideal

[markus2330]

Looks already good to me, the work on this can start. Some minor corrections.

[markus2330]

I think that for this first part it is actually best if we make an GitHub issue for every release in which we tick the things to be done (one by one). This makes it clear, e.g. if the version numbers are already incremented or not.

I created #3553

[markus2330]

Suggested change

	- [ ] Check diff of commit "Debian Package <VERSION>-1 (UNRELEASED)":
	An entry for a new upstream version should have been created in debian/changelog
	- [ ] Check diff of commit "Debian Package <VERSION>-1":
	The previously created version entry in debian/changelog should have been updated from
	UNRELEASED to unstable.

Obviously these checks are very useful for the very first time we test the release pipeline, so you can add it to some issue. But to check this every time on every release sounds too pedantic to me.

[markus2330]

Suggested change

	- Inspect the website from the pipeline
	- Inspect the website https://release.libelektra.org from the pipeline

Here obviously it will be very handy to see the URL but you can also add it later if you do not know it yet.

[markus2330]

@robaerd is this also to review?

I think this should also be merged before the release.

[robaerd]

is this also to review?

Yes please.

[markus2330]

Great work! Thank you for the very nice description of the workflow! I am looking forward to @mpranj feedback if it actually worked like this.

[markus2330]

I think we can remove this section now and directly start with testing.

[markus2330]

This is to be merged with the text below: "Run scripts/dev/update-infos-status with arguments for heuristics (--auto)"

[markus2330]

Suggested change

	== Testing ==
	== Manual Testing ==

[markus2330]

Suggested change

	If no release critical problems are found during testing, continue.

[markus2330]

Please shortly describe how to do that.

[markus2330]

Suggested change

	- [ ] Check diff of commit "Update debian/changelog for release": Changelog should have a new entry for this release
	- [ ] Check diff of commit "release: update debian/changelog": Changelog should have a new entry for this release

[markus2330]

Suggested change

	- [ ] Check diff of commit "Update fedora/changelog for release": Changelog should have a new entry for this release
	- [ ] Check diff of commit "release: update fedora/changelog": Changelog should have a new entry for this release

[markus2330]

Please add a link here.

[markus2330]

Is this verification step necessary? Maybe we can add the deb names also in the release notes and let the link checker verify?

[robaerd]

Is this verification step necessary?

The pipeline should normally fail if something would go wrong during publishing but I think it isn't a bad idea to quickly look through the repositories to briefly verify if it really was successful.

Maybe we can add the deb names also in the release notes and let the link checker verify?

I'm not sure if this would work with the link-checker since we would need to verify each (or at least one) package explicitly, e.g. https://debs.libelektra.org/bionic/pool/main/e/elektra/libelektra4-all_0.9.3-6_amd64.deb

[markus2330]

The pipeline should normally fail if something would go wrong during publishing but I think it isn't a bad idea to quickly look through the repositories to briefly verify if it really was successful.

I think we can remove this item after the next release, then.

package explicitly

Exactly, I was thinking about such a link.

[markus2330]

At least one rebuild is necessary. But it happens anyway, so isn't this step simply "waiting for master build to finish and verify website is correct".