Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[4.15-9.2] legacy-oscontainer build killed due to unexpected EOF on ppc64le #3635

Closed
aaradhak opened this issue Sep 25, 2023 · 2 comments · Fixed by #3636
Closed

[4.15-9.2] legacy-oscontainer build killed due to unexpected EOF on ppc64le #3635

aaradhak opened this issue Sep 25, 2023 · 2 comments · Fixed by #3636
Assignees
@jlebon
Labels
jira for syncing to jira

Comments

@aaradhak
Copy link
Member

aaradhak commented Sep 25, 2023 •
edited
Loading

There is a consistent build failure in legacy-oscantainer on ppc64le in [4.15-9.2] builds.

The buildextend legacy-oscontainer process gets killed due to unexpected EOF before all data is read.

Ref to the recent build job - https://jenkins-rhcos.apps.ocp-virt.prod.psi.redhat.com/job/build-arch/1915/

[2023-09-25T12:08:13.964Z] + cosa buildextend-legacy-oscontainer

''''

[2023-09-25T12:09:53.343Z] 2023-09-25 12:09:53,196 INFO - Running command: ['/usr/bin/ostree', 'checkout', '--repo', '/home/jenkins/agent/workspace/build-arch/tmp/repo', '--user-mode', '--subpath=/usr/lib/os-release', '03d279a6a0800891b03d21c3e0a20040dfdeeb5de7ace99c9fb13c3b3903be11', 'tmp/usrlib-osrelease']

[2023-09-25T12:09:54.704Z] Entering vm to build oscontainer for build: 415.92.202309231648-0

[2023-09-25T12:09:55.261Z] Config commit: 11758931931d937a2abe021309b470e39ceaf6ab

[2023-09-25T12:09:55.261Z] Using manifest: /home/jenkins/agent/workspace/build-arch/src/config/manifest-rhel-9.2.yaml

[2023-09-25T12:10:33.885Z] qemu-system-ppc64: Unexpected end-of-file before all data were read

[2023-09-25T14:52:33.667Z] Sending interrupt signal to process

[2023-09-25T14:52:33.667Z] Killing processes

[2023-09-25T14:52:33.980Z] kill finished with exit code 0

[2023-09-25T14:52:33.982Z] Sending interrupt signal to process

[2023-09-25T14:52:33.982Z] Killing processes

[2023-09-25T14:52:34.292Z] kill finished with exit code 2

[2023-09-25T14:52:45.101Z] script returned exit code 143
@aaradhak
Copy link
Member Author

Recently learned that the the issue is blocked on openshift/driver-toolkit#101 which is further blocked on https://issues.redhat.com/browse/COS-2169

@aaradhak aaradhak changed the title legacy-oscontainer build killed due to unexpected EOF on ppc64le legacy-oscontainer killed due to unexpected EOF on ppc64le [4.15-9.2] builds Sep 25, 2023
@aaradhak aaradhak changed the title legacy-oscontainer killed due to unexpected EOF on ppc64le [4.15-9.2] builds [4.15-9.2] legacy-oscontainer killed due to unexpected EOF on ppc64le builds Sep 25, 2023
@aaradhak aaradhak changed the title [4.15-9.2] legacy-oscontainer killed due to unexpected EOF on ppc64le builds [4.15-9.2] legacy-oscontainer build killed due to unexpected EOF on ppc64le Sep 25, 2023
@dustymabe
Copy link
Member

Recently learned that the the issue is blocked on openshift/driver-toolkit#101 which is further blocked on issues.redhat.com/browse/COS-2169

Yes. Removing this code path is blocked on those two linked issues. The underlying problem here is unrelated, though.

@jlebon jlebon mentioned this issue Sep 25, 2023
Closed
@jlebon jlebon added the jira for syncing to jira label Sep 26, 2023
@jlebon jlebon self-assigned this Sep 26, 2023
jlebon added a commit to jlebon/coreos-assembler that referenced this issue Sep 26, 2023
@jlebon
mantle/qemu: neuter virtiofsd seccomp filtering
5ac48fc 
By default, `virtiofsd` uses seccomp to allow only some syscalls to be
proxied from the guest. In the theme of `--sandbox=none`, let's also
neuter seccomp filtering for our virtiofs usage; the workloads we run in
the supermin/dev VMs are trusted.

Incidentally, this avoids issues like coreos#3635, where some syscalls
were accidentally missing from the allow list. In this case, new
libostree code[[1]] running in the supermin VM when building the
legacy oscontainer calls out to `fstatfs` over virtiofs, which maps to
the blocked `fstatfs64` syscall on ppc64le. (I've opened an upstream
patch[[2]] to fix this, but we don't strictly need it.)

Closes: coreos#3635

[1]: ostreedev/ostree@ba9c9de
[2]: https://gitlab.com/virtio-fs/virtiofsd/-/merge_requests/200
@jlebon jlebon mentioned this issue Sep 26, 2023
Merged
cgwalters pushed a commit that referenced this issue Sep 26, 2023
@jlebon @cgwalters
mantle/qemu: neuter virtiofsd seccomp filtering
87fc693 
By default, `virtiofsd` uses seccomp to allow only some syscalls to be
proxied from the guest. In the theme of `--sandbox=none`, let's also
neuter seccomp filtering for our virtiofs usage; the workloads we run in
the supermin/dev VMs are trusted.

Incidentally, this avoids issues like #3635, where some syscalls
were accidentally missing from the allow list. In this case, new
libostree code[[1]] running in the supermin VM when building the
legacy oscontainer calls out to `fstatfs` over virtiofs, which maps to
the blocked `fstatfs64` syscall on ppc64le. (I've opened an upstream
patch[[2]] to fix this, but we don't strictly need it.)

Closes: #3635

[1]: ostreedev/ostree@ba9c9de
[2]: https://gitlab.com/virtio-fs/virtiofsd/-/merge_requests/200
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jlebon jlebon

Labels
jira for syncing to jira
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

mantle/qemu: neuter virtiofsd seccomp filtering jlebon/coreos-assembler
3 participants
@dustymabe @jlebon @aaradhak