View Issue Details

IDProjectCategoryView StatusLast Update
0018432CentOS-8makepublic2022-04-17 17:48
Reporterrpurdie Assigned To 
PrioritynormalSeveritymajorReproducibilityrandom
Status assignedResolutionopen 
Summary0018432: make 4.2.1 sees build hangs in kernel or perl builds
DescriptionThe Yocto Project does a lot of builds on a variety of different "workers" in our CI/Autobuilder and we use different OSes there. We occasionally see hung workers and those workers appear to be Centos or a derivative and the pattern is that it is always make 4.2.1. We usually see make just sitting with no build processes active.

You can see some debugging I did of this with a hung perl build here: https://lists.yoctoproject.org/g/swat/topic/info_for_swat_for_hung_build/86698083
We have https://bugzilla.yoctoproject.org/show_bug.cgi?id=14712 tracking some of the kernel build issues on our side.

I strongly suspect the issue is:

https://git.savannah.gnu.org/cgit/make.git/commit/?id=78b5fec6898c26956d00548427cda1101cb80f8a
https://savannah.gnu.org/bugs/?51400

which is fixed in make 4.3.

I'd also mention this was fixed in ubuntu make 4.2.1 back in 2018:

https://ubuntu.pkgs.org/20.04/ubuntu-main-arm64/make_4.2.1-1.2_arm64.deb.html

"""* Fix potential hangs (Closes: #890309):
- [SV 51159] Use a non-blocking read with pselect to avoid hangs.
- [SV 51400] Only unblock fatal signals after child invocation"""

I was wondering if you'd consider adding the patch for this issue?

Steps To ReproduceWe do not have a way to reproduce this at will unfortunately.
TagsNo tags attached.

Activities

toracat

toracat

2022-04-14 21:12

manager   ~0038901

Could you provide the precise version of make? The current version in EL8 is make-4.2.1-10.el8.

make-4.2.1-9.el8 has a known bug (1), which may or may not be related to the issue you are reporting.

(1) https://bugzilla.redhat.com/show_bug.cgi?id=1774790
halstead

halstead

2022-04-14 22:48

reporter   ~0038903

In case it is helpful:

Alma8 workers had make-1:4.2.1-10 on install and no upgrades since.

Stream got make-1:4.2.1-11 at install and no upgrades.

CentOS8 machines got make-1:4.2.1-9 on install 2020-06-26, make-1:4.2.1-10 on first update 2020-06-26, and make-1:4.2.1-10 on 2021-10-29.
rpurdie

rpurdie

2022-04-15 09:58

reporter   ~0038909

Thanks halstead! Since the failures have occurred since 2021-10-29 we would be using 1:4.2.1-10. If some of those fixes were in -10, there must be some other difference in patchset between this and the ubuntu 4.2.1 version since we've never seen this issue on an ubuntu worker and they all run the same workloads.
toracat

toracat

2022-04-17 17:48

manager   ~0038912

CentOS-8 reached EOL on Dec 31, 2021, so no more fix.

Since the bug you are reporting exists in CentOS Stream, you need to report this to Red Hat. For details, please see:

https://wiki.centos.org/ReportBugs

While I and my colleagu regularly build the kernel on RHEL-8 and (in the past CentOS-8 as well), we've never encountered the problem described here. So when filing a bug upstream, make sure you provide ample information as to how to reproduce the issue.

Issue History

Date Modified Username Field Change
2022-04-13 15:05 rpurdie New Issue
2022-04-14 21:12 toracat Note Added: 0038901
2022-04-14 21:13 toracat Status new => feedback
2022-04-14 22:48 halstead Note Added: 0038903
2022-04-15 09:58 rpurdie Note Added: 0038909
2022-04-15 09:58 rpurdie Status feedback => assigned
2022-04-17 17:48 toracat Note Added: 0038912