Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes silent package sync uploading failures #158

Merged
merged 1 commit into from
Jul 16, 2019
Merged

Conversation

eeyun
Copy link
Contributor

@eeyun eeyun commented Jul 9, 2019

This change fixes the silent upload failure and adds the hab-sup and hab-launcher to the base-plans only syncing mechanism.
Signed-off-by: Ian Henry [email protected]

@chefsalim
Copy link
Contributor

Nice! For posterity it would be good to add more explanation of the breakage and changes.

@eeyun eeyun force-pushed the eeyun/sync_fixes branch 4 times, most recently from 27c4763 to 5238e50 Compare July 10, 2019 17:13
@eeyun
Copy link
Contributor Author

eeyun commented Jul 10, 2019

Ok, so, let me break down why this change exists. First, this fixes #153 .

Second it turned out there were a couple of silent failure cases in the sync script. At some point in the last few months we changes the API for upload endpoints to need a package's target. Via the hab cli this is never a problem since the package target is a bit of metadata the client can glean and pass along on upload. Since the sync script needs to be able to upload the full set while avoiding the "this package's deps don't exist" error, we use curl. Doing so means the script had a hand rolled POST statement and we needed to update the statement to include the package target.

Next, I found that in cases where a distro has a b2sum binary that ships with it, the flags from the go binary we referenced in the docs did not match the flags from the binary shipped by blake2 themselves. As such it was possible for a second silent failure in which the incorrect flag would be passed to generate the package checksum.

Now, we use the default flags from the common binary AND any of the binary utilities we use in the script are now explicitly executed as habitat packages rather than relying on the underlying distro.

@eeyun eeyun force-pushed the eeyun/sync_fixes branch from 5238e50 to 9023804 Compare July 10, 2019 17:16
@@ -220,6 +228,10 @@ read_base_plans() {
done < base-plans.txt
}

read_hab_plans() {
hab_plans=( "hab-sup" "hab-launcher")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this also cover hab-backline as an automatic dependency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will for a full sync. This stanza is only used with the base-plans flag.

@eeyun eeyun force-pushed the eeyun/sync_fixes branch from 9023804 to e55763f Compare July 10, 2019 20:49
@eeyun eeyun force-pushed the eeyun/sync_fixes branch from e55763f to 58d30d4 Compare July 11, 2019 16:22
@eeyun eeyun force-pushed the eeyun/sync_fixes branch from 58d30d4 to 5f98201 Compare July 12, 2019 14:10
Copy link

@scotthain scotthain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should add a "has sudo" check, as well as the comment i left.

scripts/on-prem-archive.sh Show resolved Hide resolved
@eeyun eeyun force-pushed the eeyun/sync_fixes branch from 5f98201 to 4f22085 Compare July 16, 2019 14:08
@eeyun eeyun merged commit c20f4ba into master Jul 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants