[Bug] ValueError: SHA could not be resolved, git returned: b''

[HebaruSan]

Problem

The metadata tester is broken. It now emits this every time it runs:

Traceback (most recent call last):
  File "/usr/local/bin/ckanmetatester", line 11, in <module>
    load_entry_point('CkanMetaTester==0.1', 'console_scripts', 'ckanmetatester')()
  File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/__init__.py", line 24, in test_metadata
    environ.get('INPUT_DIFF_META_ROOT'))
  File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 74, in test_metadata
    for file in self.files_to_test(source):
  File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 231, in files_to_test
    return self.paths_from_diff(self.branch_diff(Repo('.')))
  File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 241, in branch_diff
    logging.debug('Looking for merge base between %s and %s', start_ref, repo.head.commit.hexsha)
  File "/usr/local/lib/python3.7/dist-packages/git/refs/symbolic.py", line 226, in _get_commit
    obj = self._get_object()
  File "/usr/local/lib/python3.7/dist-packages/git/refs/symbolic.py", line 219, in _get_object
    return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
  File "/usr/local/lib/python3.7/dist-packages/git/objects/base.py", line 94, in new_from_sha
    oinfo = repo.odb.info(sha1)
  File "/usr/local/lib/python3.7/dist-packages/git/db.py", line 40, in info
    hexsha, typename, size = self._git.get_object_header(bin_to_hex(binsha))
  File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line [13](https://github.com/KSP-CKAN/NetKAN/actions/runs/3895070850/jobs/6678636081#step:6:14)83, in get_object_header
    return self.__get_object_header(cmd, ref)
  File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line 1370, in __get_object_header
    return self._parse_object_header(cmd.stdout.readline())
  File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line 13[29](https://github.com/KSP-CKAN/NetKAN/actions/runs/3895070850/jobs/6678636081#step:6:30), in _parse_object_header
    raise ValueError("SHA could not be resolved, git returned: %r" % (header_line.strip()))
ValueError: SHA could not be resolved, git returned: b''

It worked on this commit:

• KSP-CKAN/NetKAN@9f69821

And broke on this one, one minute later:

• KSP-CKAN/NetKAN@2790b37

In between, the docker image was regenerated:

• https://hub.docker.com/repository/docker/kspckan/metadata/general

... as a result of this commit:

KSP-CKAN/CKAN@d011444

So presumably something in the dependencies changed. There was a GitPython release 3 weeks ago, but reading the main issue it fixed didn't reveal much to me:

• CVE-2022-24439: <gitpython::clone> 'ext::sh -c touch% /tmp/pwned' for remote code execution gitpython-developers/GitPython#1515
• Block insecure options and protocols by default gitpython-developers/GitPython#1521

If there's a connection between that and repo.head.commit.hexsha no longer working, it eludes me.

[HebaruSan]

Maybe the repo.head property was removed? I see no trace of it in the docs so far:

https://gitpython.readthedocs.io/en/stable/reference.html

Never mind, it's there:

https://gitpython.readthedocs.io/en/stable/reference.html#git.repo.base.Repo.head

And it looks like the rest of the properties should still work as well:

https://gitpython.readthedocs.io/en/stable/reference.html#git.refs.head.Head

[HebaruSan]

Found same stack trace in gitpython-developers/GitPython#1121 / gitpython-developers/GitPython#1180

... which unfortunately concluded without identifying a cause or solution, and instead kind of petered out with something resembling a workaround or the start of an investigation.

[HebaruSan]

Link for easy auditing of the code that throws:

https://github.com/gitpython-developers/GitPython/blob/main/git/cmd.py

[HebaruSan]

gitpython-developers/GitPython#1016 looks also related and possibly more enlightening:

I was getting this error message when I was trying to get info about a repository which I was not the owner of (even though I was root). I tried doing it manually with git rev-parse HEAD and got
fatal: unsafe repository ('[my repo]' is owned by someone else)

If you're having this problem, see if git rev-parse HEAD works.

I would like to see some better error handling if possible. It seems clear to me that an error was not properly being raised somewhere and a function was silently returning an empty binary string.

Maybe git is running into a different owner than the repository.

a recent update to Git on the apt repositories added a feature which is causing this problem.

This suggests a two-part problem; first, git itself hypothetically fails due to some obscure permissions checking issue (this being the proximate cause, some new feature since the last time our docker image was rebuilt), then GitPython fails by discarding all the information it receives about that problem and simply returning an empty string, confusing its own internals.

https://stackoverflow.com/a/71904131/3580553

[HebaruSan]

Since that hypothesis implicates permissions, here's where our workflows are configured:

• https://github.com/KSP-CKAN/NetKAN/blob/master/.github/workflows/inflate.yml
• https://github.com/KSP-CKAN/CKAN-meta/blob/master/.github/workflows/validate.yml
• https://github.com/KSP-CKAN/xKAN-meta_testing/blob/master/action.yml
• https://github.com/KSP-CKAN/CKAN/blob/master/Dockerfile.metadata

So far I see no indication that we're doing anything weird, but who knows what might be going on all these GitHub Actions...

[HebaruSan]

actions/checkout seems to have been updated to run the "safe directory" setting itself:

• https://github.com/actions/checkout

    # Add repository path as safe.directory for Git global config by running `git
    # config --global --add safe.directory <path>`
    # Default: true
    set-safe-directory: ''

• Enforce safe directory actions/checkout#762
• Add set-safe-directory input to allow customer to take control. actions/checkout#770

It says the default is true, so it seems like that should already be in effect, but maybe we need to use a newer version?

EDIT: Yup. That's for v3, and we're using v2!

EDIT 2: Nope, v2 was already running the safe directory command. Maybe they did a hotfix.

[HebaruSan]

Ohhhhh. actions/checkout#762:

This does carry some limitations, it doesn't persist this configuration for the duration of the job mainly. So if your job pushes to git or something after checkout, that will continue to fail.

All they were fixing was the checkout action itself failing. Downstream users may still be affected.

[infor7]

When I set up HOME env to my user home directory the error disappear:
export HOME=/home/username

[HebaruSan]

#90 seems to have fixed this, based on KSP-CKAN/NetKAN#9513's validation now having succeeded.

[linuxgurugamer]

Might I suggest that you set up a backup of the validator image so that if it ever breaks again, you can revert to the previous version?