Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: valid SOURCE_DATE_EPOCH #1520

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

xnox
Copy link
Contributor

@xnox xnox commented Sep 27, 2024

Set the floor of SOURCE_DATE_EPOCH to 1980, as ZIP file format does
not support timestamps prior to 1980, meaning zip, jar, egg, wheel all
fail to be created or used.

Without this fix, if one removes the hand override of SOURCE_DATE_EPOCH from py3-userpath, the build fails:

melange build py3-userpath.yaml --arch x86_64  --pipeline-dir ./pipelines/ -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub -r https://packages.wolfi.dev/os --source-dir ./py3-userpath/
...
2024/09/27 14:55:33 INFO * Building wheel... name="Python Build"
2024/09/27 14:55:33 WARN Traceback (most recent call last): name="Python Build"
2024/09/27 14:55:33 WARN   File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module> name="Python Build"
2024/09/27 14:55:33 WARN     main() name="Python Build"
2024/09/27 14:55:33 WARN   File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main name="Python Build"
2024/09/27 14:55:33 WARN     json_out["return_val"] = hook(**hook_input["kwargs"]) name="Python Build"
2024/09/27 14:55:33 WARN                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 271, in build_wheel name="Python Build"
2024/09/27 14:55:33 WARN     return _build_backend().build_wheel( name="Python Build"
2024/09/27 14:55:33 WARN            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/tmp/build-env-u9wepgmk/lib/python3.12/site-packages/hatchling/build.py", line 58, in build_wheel name="Python Build"
2024/09/27 14:55:33 WARN     return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard']))) name="Python Build"
2024/09/27 14:55:33 WARN                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/tmp/build-env-u9wepgmk/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 155, in build name="Python Build"
2024/09/27 14:55:33 WARN     artifact = version_api[version](directory, **build_data) name="Python Build"
2024/09/27 14:55:33 WARN                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/tmp/build-env-u9wepgmk/lib/python3.12/site-packages/hatchling/builders/wheel.py", line 476, in build_standard name="Python Build"
2024/09/27 14:55:33 WARN     record = archive.add_file(included_file) name="Python Build"
2024/09/27 14:55:33 WARN              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/tmp/build-env-u9wepgmk/lib/python3.12/site-packages/hatchling/builders/wheel.py", line 98, in add_file name="Python Build"
2024/09/27 14:55:33 WARN     zip_info = zipfile.ZipInfo(relative_path, cast(TIME_TUPLE, self.time_tuple)) name="Python Build"
2024/09/27 14:55:33 WARN                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/usr/lib/python3.12/zipfile/__init__.py", line 398, in __init__ name="Python Build"
2024/09/27 14:55:33 WARN     raise ValueError('ZIP does not support timestamps before 1980') name="Python Build"
2024/09/27 14:55:33 WARN ValueError: ZIP does not support timestamps before 1980 name="Python Build"
2024/09/27 14:55:33 INFO name="Python Build"
2024/09/27 14:55:33 INFO ERROR Backend subprocess exited when trying to invoke build_wheel name="Python Build"
2024/09/27 14:55:33 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/09/27 14:55:33 INFO   workspace dir: /tmp/melange-workspace-1020228742
2024/09/27 14:55:33 INFO   guest dir: /tmp/melange-guest-609671933
2024/09/27 14:55:33 ERRO failed to build package: unable to run package py3-userpath pipeline: unable to run pipeline: exit status 1

With this fix inplace it just builds correctly. This will remove many
custom overrides of SOURCE_DATE_EPOCH in wolfi-dev/os, and prevent any
developer hitting such pitfall.

This is similar to what was done in NixOS, Debian, Python and probably
other projects.

In Wolfi this will fix building brand new packages, and building
things directly with melange/wolfictl outside of pipelines and
makefiles.

See for example NixOS/nixpkgs#89794

Set the floor of SOURCE_DATE_EPOCH to 1980, as ZIP file format does
not support timestamps prior to 1980, meaning zip, jar, egg, wheel all
fail to be created or used.

Without this fix, if one removes the hand override of SOURCE_DATE_EPOCH from py3-userpath, the build fails:

```console
melange build py3-userpath.yaml --arch x86_64  --pipeline-dir ./pipelines/ -k https://packages.wolfi.dev/os/wolfi-signing.rsa.pub -r https://packages.wolfi.dev/os --source-dir ./py3-userpath/
...
2024/09/27 14:55:33 INFO * Building wheel... name="Python Build"
2024/09/27 14:55:33 WARN Traceback (most recent call last): name="Python Build"
2024/09/27 14:55:33 WARN   File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module> name="Python Build"
2024/09/27 14:55:33 WARN     main() name="Python Build"
2024/09/27 14:55:33 WARN   File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main name="Python Build"
2024/09/27 14:55:33 WARN     json_out["return_val"] = hook(**hook_input["kwargs"]) name="Python Build"
2024/09/27 14:55:33 WARN                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 271, in build_wheel name="Python Build"
2024/09/27 14:55:33 WARN     return _build_backend().build_wheel( name="Python Build"
2024/09/27 14:55:33 WARN            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/tmp/build-env-u9wepgmk/lib/python3.12/site-packages/hatchling/build.py", line 58, in build_wheel name="Python Build"
2024/09/27 14:55:33 WARN     return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard']))) name="Python Build"
2024/09/27 14:55:33 WARN                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/tmp/build-env-u9wepgmk/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 155, in build name="Python Build"
2024/09/27 14:55:33 WARN     artifact = version_api[version](directory, **build_data) name="Python Build"
2024/09/27 14:55:33 WARN                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/tmp/build-env-u9wepgmk/lib/python3.12/site-packages/hatchling/builders/wheel.py", line 476, in build_standard name="Python Build"
2024/09/27 14:55:33 WARN     record = archive.add_file(included_file) name="Python Build"
2024/09/27 14:55:33 WARN              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/tmp/build-env-u9wepgmk/lib/python3.12/site-packages/hatchling/builders/wheel.py", line 98, in add_file name="Python Build"
2024/09/27 14:55:33 WARN     zip_info = zipfile.ZipInfo(relative_path, cast(TIME_TUPLE, self.time_tuple)) name="Python Build"
2024/09/27 14:55:33 WARN                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ name="Python Build"
2024/09/27 14:55:33 WARN   File "/usr/lib/python3.12/zipfile/__init__.py", line 398, in __init__ name="Python Build"
2024/09/27 14:55:33 WARN     raise ValueError('ZIP does not support timestamps before 1980') name="Python Build"
2024/09/27 14:55:33 WARN ValueError: ZIP does not support timestamps before 1980 name="Python Build"
2024/09/27 14:55:33 INFO name="Python Build"
2024/09/27 14:55:33 INFO ERROR Backend subprocess exited when trying to invoke build_wheel name="Python Build"
2024/09/27 14:55:33 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/09/27 14:55:33 INFO   workspace dir: /tmp/melange-workspace-1020228742
2024/09/27 14:55:33 INFO   guest dir: /tmp/melange-guest-609671933
2024/09/27 14:55:33 ERRO failed to build package: unable to run package py3-userpath pipeline: unable to run pipeline: exit status 1
```

With this fix inplace it just builds correctly. This will remove many
custom overrides of SOURCE_DATE_EPOCH in wolfi-dev/os, and prevent any
developer hitting such pitfall.

This is similar to what was done in NixOS, Debian, Python and probably
other projects.

In Wolfi this will fix building brand new packages, and building
things directly with melange/wolfictl outside of pipelines and
makefiles.

See for example NixOS/nixpkgs#89794
@xnox
Copy link
Contributor Author

xnox commented Sep 27, 2024

Once this is in wolfictl, we will be able to eliminate

$ git grep SOURCE_DATE_EPOCH *.yaml | grep 3155 | wc -l
48

At least that many lines of yaml, from wolfi (and often 3x that because of a comment, and environment: block, and human having had spent time writing that).

imjasonh
imjasonh previously approved these changes Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants