-
Notifications
You must be signed in to change notification settings - Fork 280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
libpkg: rework internal ABI handling, fix bugs #2376
Conversation
First of all, apologies for the scope creep on this one. I didn't find a good way to break it up into smaller patches. Writing this was a bit like pulling a thread that caused the whole knit sweater to unravel. This is in my opinion a much needed refactor that will make future work on further architecture-specific code such as the shlib tracking features significantly easier. The main guiding principle of this patch is to create a single source of truth for as many ABI related details as possible. I'll summarize the changes with a series of before/after comparisons: Before: Setting -o ABI=... on the command line but not -o OSVERSION=... led to inconsistency between ABI and OSVERSION. After: Setting ABI requires setting OSVERSION as well (and vice versa). (Note: OSVERSION is FreeBSD-specific) Related: freebsd#2363 Before: ALTABI set in the environment, with -o, or in pkg.conf only affected the ALTABI variable expanded while parsing repo conf files. After: setting ALTABI is completely ignored and a notice is printed if it is set. The ALTABI formatted string derived from the automatically detected or configured ABI is still printed by `pkg config ALTABI`. Before: Setting ABI/OSVERSION in pkg.conf would not affect the variable expansions of ABI, OSVERSION, ARCH, VERSION_MAJOR, etc. in pkg.conf. This was a pretty big footgun IMO, though I don't know of any examples of people getting bitten by this. I don't think setting ABI in pkg.conf is a normal thing to do. After: Setting ABI, ALTABI, and OSVERSION in pkg.conf is not allowed. This is consistent with ABI_FILE, which was never able to be set using pkg.conf. It is still possible to set ABI and OSVERSION on the command line and e.g. `pkg config ABI` still works. This is the simplest solution I see to this problem, but it is a breaking change. Before: Parsing of user-provided ABI strings was very lax. Pretty much anything was accepted without giving any error as long is it contained at least one colon. The full affects of providing such invalid ABI strings to pkg are not well understood, I think it can safely be assumed that interesting bugs would result. After: Parsing of user-provided ABI strings is very strict. Only a well defined set of supported OS names and architectures are accepted. Missing or extra components cause a hard error. Before: The global `ctx.oi` os_info struct stored multiple copies of the same information that could get out of sync with each other. For example, the version was stored in 4 or 5 different places in the same struct. After: The global `ctx.abi` struct contains no redundant information. Before: Most code dealing with ABI information worked with stringly typed data. Conversion to/from strings and parsing details were strewn across the codebase. After: Most code dealing with ABI information uses the new `struct pkg_abi` representation with enums for OS and architecture and integers for the version. All parsing is unified in pkg_abi.c and multiple minor inconsistencies between previously duplicated code are resolved. Closes: freebsd#2363 Sponsored by: The FreeBSD Foundation
4c3fd2f
to
945a844
Compare
@kevemueller I did end up leaving the It would be trivial to add them or any other currently unsupported architecture to that enum and the handful of functions converting to/from strings in pkg_abi.c if/when pkg gains full support for new architectures. |
I realized that we do have an alternative here that isn't terribly complex to implement. We could parse
Since we would still need to error out if ABI is set but not OSVERSION I'm not sure if this would make this PR as a whole any less breaking in practice. I'd be happy to implement this alternative if it would be preferable to continue to be able to set ABI/OSVERSION from pkg.conf. |
@@ -343,7 +271,7 @@ analyse_elf(struct pkg *pkg, const char *fpath) | |||
goto cleanup; /* not a dynamically linked elf: no results */ | |||
} | |||
|
|||
if (!shlib_valid_abi(fpath, &elfhdr)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change here nicely highlights my motivation for working on this patch. Having a standard, non-textual representation of the ABI lets us get rid of the entire shlib_vaild_abi()
function and use the same elf_parse_arch()
function we use to automatically detect the ABI from a file.
The best part is that this is a bug fix as well as a simplification, shlib_valid_abi()
currently tries to parse the ABI string as if it had the ALTABI format and ends always returning true in practice regardless of the input.
This change will make implementing #2331 far easier.
rebased and merged manually |
First of all, apologies for the scope creep on this one. I didn't find a good way to break it up into smaller patches. Writing this was a bit like pulling a thread that caused the whole knit sweater to unravel.
This is in my opinion a much needed refactor that will make future work on further architecture-specific code such as the shlib tracking features significantly easier.
The main guiding principle of this patch is to create a single source of truth for as many ABI related details as possible. I'll summarize the changes with a series of before/after comparisons:
Before: Setting -o ABI=... on the command line but not -o OSVERSION=... led to inconsistency between ABI and OSVERSION.
After: Setting ABI requires setting OSVERSION as well (and vice versa). (Note: OSVERSION is FreeBSD-specific)
Related: #2363
Before: ALTABI set in the environment, with -o, or in pkg.conf only affected the ALTABI variable expanded while parsing repo conf files and nothing else.
After: Setting ALTABI is completely ignored and a notice is printed if it is set. The ALTABI formatted string derived from the automatically detected or configured ABI is still printed by
pkg config ALTABI
.Before: Setting ABI/OSVERSION in pkg.conf would not affect the variable expansions of ABI, OSVERSION, ARCH, VERSION_MAJOR, etc. in pkg.conf. This was a pretty big footgun IMO, though I don't know of any examples of people getting bitten by this. I don't think setting ABI in pkg.conf is a normal thing to do.
After: Setting ABI, ALTABI, and OSVERSION in pkg.conf is not allowed. This is consistent with ABI_FILE, which was never able to be set using pkg.conf. It is still possible to set ABI and OSVERSION on the command line and e.g.
pkg config ABI
still works. This is the simplest solution I see to this problem, but it is a breaking change.Before: Parsing of user-provided ABI strings was very lax. Pretty much anything was accepted without giving any error as long is it contained at least one colon. The full affects of providing such invalid ABI strings to pkg are not well understood, I think it can safely be assumed that interesting bugs would result.
After: Parsing of user-provided ABI strings is very strict. Only a well defined set of supported OS names and architectures are accepted. Missing or extra components cause a hard error.
Before: The global
ctx.oi
os_info struct stored multiple copies of the same information that could get out of sync with each other. For example, the version was stored in 4 or 5 different places in the same struct.After: The global
ctx.abi
struct contains no redundant information.Before: Most code dealing with ABI information worked with stringly typed data. Conversion to/from strings and parsing details were strewn across the codebase.
After: Most code dealing with ABI information uses the new
struct pkg_abi
representation with enums for OS and architecture and integers for the version. All parsing is unified in pkg_abi.c and multiple minor inconsistencies between previously duplicated code are resolved.