|
|
see also: https://www.hivelocity.net/blog/centos-update-advisory/ |
|
|
|
Correct, I am apart of the Company Hivelocity. |
|
|
|
You power cycled the machine while it was in mid-update, specifically while the microcode_ctl package was being updated. If you do this then the new kernel that was just installed is not complete as the initramfs for the newly installed kernel is not generated until the cleanup cycle of the yum update process. So by power cycling the machine you have half installed the new kernel and there is no valid initramfs file for it so it will not boot. Now, *why* the microcode_ctl update appeared to be hung is a different question but power cycling the machine in mid-update is pretty much guaranteed to mess things up badly. You probably have a whole bunch of duplicate packages installed as well.
Does this work if you exclude microcode_ctl from the update process? |
|
|
|
I am adding a note to second this issue. I have tried on 2 separate servers with the same result. The final time I left the server more than an hour trying to install microcode_ctl. I am also using Supermicro motherboards but with a Xeon E3-1585V5 processor. |
|
|
|
I have just done a clean install. Excluded microcode_ctl and it all worked fine. |
|
|
|
tinpotnick: thanks for the update.
Does the machine freeze also when you only install the microcode_ctl package after all the other upgrade are applied? |
|
|
|
I believe microcode_ctl has scripts which attempt to activate the new microcode as part of the install. That's why it's hung. It would do the same thing on reboot since the package has been installed, even though it's not yet complete. The weird error about /bin/init not being the correct format executable is almost certainly due to the power cycle and an incompletely installed kernel/initramfs but even if it wasn't, it would probably hang on reboot when it got to trying to apply the broken microcode.
For now, the best thing that anyone with the affected processors can do is to exclude this microcode_ctl package from the update. |
|
|
|
... and file bugs at RedHat, Intel and Supermicro |
|
|
|
Thanks for all the info all, I can leave the server alone for hours and it will not go anywhere, I will setup a fresh install and exclude microcode_ctl in exclude= in /etc/yum.conf, run the update, see if it hangs and let the updates complete for microcode_ctl after removing, per tru. i will start working on filing those cases elsewhere as well and will let you all know the headway I make.
Thanks! |
|
|
|
@dbennett: it was reported that the update worked if used as rpm -U --noscripts. maybe you can try that as well ? |
|
|
|
I also encountered the same problem. My motherboard was X11SSL-F. Even worse, after the update, my RAID data was damaged and many files became 0byte. I wanted to cry when I saw it. |
|
|
|
I encountered this issue yesterday on HPE ProLiant ML10 Gen9 Server running CentOS release 7.8.2003 with Linux 3.10.0-1127.10.1.el7.x86_64 kernel. Fortunately the microcode_ctl package was the only update involved so my path to recovery after hard shut down was:
1. Used grub editor during boot to include dis_ucode_ldr in boot arguments
2. Entered ctrl-x from grub editor to continue boot
3. yum remove microcode_ctl (note that this removed microcode_ctl 2.1-61.6.el7-8)
4. yum install microcode_ctl-2.1-61.el7
5. Subsequent shut down and boot up was then normal |
|
|
|
robetek, I am rather unlucky. grub.cfg becomes 0byte, and cannot be booted. The rules of selinux are all changed to 0byte, and many configuration files have become 0byte. Use the previous day's backup to restore. I thought the HDD in the RAID was broken. After replacing the HDD with another one, yum update was done again, and the RAID was damaged again. I realized that it was not the HDD. |
|
|
|
After getting a clean install of CentOS 7 going, I excluded microcode_ctl from /etc/yum.conf and ran updates successfully, removed it after the reboot, updated, the server hung but after 2 minutes, it rebooted and is online.
It seems that something while its in the middle of updating all the other packages causes this one to hang, I will work on giving rpm -U --noscripts a try in a bit per @ManuelWolfshant
history.txt (25,839 bytes)
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ssh root@23.111.168.58
root@23.111.168.58's password:
[root@X11TESTSERVER ~]# vi /etc/yum.conf
[root@X11TESTSERVER ~]# yum update -y
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Determining fastest mirrors
* base: mirror.netdepot.com
* extras: mirror.teklinks.com
* updates: mirror.mojohost.com
base | 3.6 kB 00:00:00
extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/4): base/7/x86_64/group_gz | 153 kB 00:00:00
(2/4): updates/7/x86_64/primary_db | 2.1 MB 00:00:00
(3/4): extras/7/x86_64/primary_db | 194 kB 00:00:00
(4/4): base/7/x86_64/primary_db | 6.1 MB 00:00:00
Resolving Dependencies
--> Running transaction check
---> Package bind-export-libs.x86_64 32:9.11.4-16.P2.el7 will be updated
---> Package bind-export-libs.x86_64 32:9.11.4-16.P2.el7_8.6 will be an update
---> Package binutils.x86_64 0:2.27-43.base.el7 will be updated
---> Package binutils.x86_64 0:2.27-43.base.el7_8.1 will be an update
---> Package device-mapper.x86_64 7:1.02.164-7.el7 will be updated
---> Package device-mapper.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-event.x86_64 7:1.02.164-7.el7 will be updated
---> Package device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-event-libs.x86_64 7:1.02.164-7.el7 will be updated
---> Package device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-libs.x86_64 7:1.02.164-7.el7 will be updated
---> Package device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package firewalld.noarch 0:0.6.3-8.el7 will be updated
---> Package firewalld.noarch 0:0.6.3-8.el7_8.1 will be an update
---> Package firewalld-filesystem.noarch 0:0.6.3-8.el7 will be updated
---> Package firewalld-filesystem.noarch 0:0.6.3-8.el7_8.1 will be an update
---> Package iprutils.x86_64 0:2.4.17.1-3.el7 will be updated
---> Package iprutils.x86_64 0:2.4.17.1-3.el7_7 will be an update
---> Package kernel.x86_64 0:3.10.0-1127.10.1.el7 will be installed
---> Package kernel-tools.x86_64 0:3.10.0-1127.el7 will be updated
---> Package kernel-tools.x86_64 0:3.10.0-1127.10.1.el7 will be an update
---> Package kernel-tools-libs.x86_64 0:3.10.0-1127.el7 will be updated
---> Package kernel-tools-libs.x86_64 0:3.10.0-1127.10.1.el7 will be an update
---> Package lvm2.x86_64 7:2.02.186-7.el7 will be updated
---> Package lvm2.x86_64 7:2.02.186-7.el7_8.2 will be an update
---> Package lvm2-libs.x86_64 7:2.02.186-7.el7 will be updated
---> Package lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 will be an update
---> Package python-firewall.noarch 0:0.6.3-8.el7 will be updated
---> Package python-firewall.noarch 0:0.6.3-8.el7_8.1 will be an update
---> Package python-perf.x86_64 0:3.10.0-1127.el7 will be updated
---> Package python-perf.x86_64 0:3.10.0-1127.10.1.el7 will be an update
---> Package systemd.x86_64 0:219-73.el7.1 will be updated
---> Package systemd.x86_64 0:219-73.el7_8.6 will be an update
---> Package systemd-libs.x86_64 0:219-73.el7.1 will be updated
---> Package systemd-libs.x86_64 0:219-73.el7_8.6 will be an update
---> Package systemd-sysv.x86_64 0:219-73.el7.1 will be updated
---> Package systemd-sysv.x86_64 0:219-73.el7_8.6 will be an update
---> Package tzdata.noarch 0:2019c-1.el7 will be updated
---> Package tzdata.noarch 0:2020a-1.el7 will be an update
---> Package yum-plugin-fastestmirror.noarch 0:1.1.31-53.el7 will be updated
---> Package yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================= Package Arch Version Repository Size
=========================================================================================================================Installing:
kernel x86_64 3.10.0-1127.10.1.el7 updates 50 M
Updating:
bind-export-libs x86_64 32:9.11.4-16.P2.el7_8.6 updates 1.1 M
binutils x86_64 2.27-43.base.el7_8.1 updates 5.9 M
device-mapper x86_64 7:1.02.164-7.el7_8.2 updates 295 k
device-mapper-event x86_64 7:1.02.164-7.el7_8.2 updates 191 k
device-mapper-event-libs x86_64 7:1.02.164-7.el7_8.2 updates 190 k
device-mapper-libs x86_64 7:1.02.164-7.el7_8.2 updates 324 k
firewalld noarch 0.6.3-8.el7_8.1 updates 443 k
firewalld-filesystem noarch 0.6.3-8.el7_8.1 updates 51 k
iprutils x86_64 2.4.17.1-3.el7_7 updates 243 k
kernel-tools x86_64 3.10.0-1127.10.1.el7 updates 8.0 M
kernel-tools-libs x86_64 3.10.0-1127.10.1.el7 updates 8.0 M
lvm2 x86_64 7:2.02.186-7.el7_8.2 updates 1.3 M
lvm2-libs x86_64 7:2.02.186-7.el7_8.2 updates 1.1 M
python-firewall noarch 0.6.3-8.el7_8.1 updates 354 k
python-perf x86_64 3.10.0-1127.10.1.el7 updates 8.0 M
systemd x86_64 219-73.el7_8.6 updates 5.1 M
systemd-libs x86_64 219-73.el7_8.6 updates 416 k
systemd-sysv x86_64 219-73.el7_8.6 updates 94 k
tzdata noarch 2020a-1.el7 updates 495 k
yum-plugin-fastestmirror noarch 1.1.31-54.el7_8 updates 34 k
Transaction Summary
=========================================================================================================================Install 1 Package
Upgrade 20 Packages
Total download size: 92 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
warning: /var/cache/yum/x86_64/7/updates/packages/device-mapper-event-1.02.164-7.el7_8.2.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Public key for device-mapper-event-1.02.164-7.el7_8.2.x86_64.rpm is not installed
(1/21): device-mapper-event-1.02.164-7.el7_8.2.x86_64.rpm | 191 kB 00:00:00
(2/21): binutils-2.27-43.base.el7_8.1.x86_64.rpm | 5.9 MB 00:00:00
(3/21): device-mapper-1.02.164-7.el7_8.2.x86_64.rpm | 295 kB 00:00:00
(4/21): bind-export-libs-9.11.4-16.P2.el7_8.6.x86_64.rpm | 1.1 MB 00:00:00
(5/21): device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64.rpm | 190 kB 00:00:00
(6/21): firewalld-filesystem-0.6.3-8.el7_8.1.noarch.rpm | 51 kB 00:00:00
(7/21): device-mapper-libs-1.02.164-7.el7_8.2.x86_64.rpm | 324 kB 00:00:00
(8/21): iprutils-2.4.17.1-3.el7_7.x86_64.rpm | 243 kB 00:00:00
(9/21): firewalld-0.6.3-8.el7_8.1.noarch.rpm | 443 kB 00:00:00
(10/21): kernel-tools-libs-3.10.0-1127.10.1.el7.x86_64.rpm | 8.0 MB 00:00:00
(11/21): kernel-tools-3.10.0-1127.10.1.el7.x86_64.rpm | 8.0 MB 00:00:00
(12/21): lvm2-2.02.186-7.el7_8.2.x86_64.rpm | 1.3 MB 00:00:00
(13/21): python-firewall-0.6.3-8.el7_8.1.noarch.rpm | 354 kB 00:00:00
(14/21): lvm2-libs-2.02.186-7.el7_8.2.x86_64.rpm | 1.1 MB 00:00:00
(15/21): python-perf-3.10.0-1127.10.1.el7.x86_64.rpm | 8.0 MB 00:00:00
(16/21): systemd-libs-219-73.el7_8.6.x86_64.rpm | 416 kB 00:00:00
(17/21): systemd-sysv-219-73.el7_8.6.x86_64.rpm | 94 kB 00:00:00
(18/21): systemd-219-73.el7_8.6.x86_64.rpm | 5.1 MB 00:00:00
(19/21): yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch.rpm | 34 kB 00:00:00
(20/21): tzdata-2020a-1.el7.noarch.rpm | 495 kB 00:00:00
(21/21): kernel-3.10.0-1127.10.1.el7.x86_64.rpm | 50 MB 00:00:02
-------------------------------------------------------------------------------------------------------------------------Total 33 MB/s | 92 MB 00:00:02
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-8.2003.0.el7.centos.x86_64 (@anaconda)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : systemd-libs-219-73.el7_8.6.x86_64 1/41
Updating : systemd-219-73.el7_8.6.x86_64 2/41
Updating : 7:device-mapper-1.02.164-7.el7_8.2.x86_64 3/41
Updating : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64 4/41
Updating : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64 5/41
Updating : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64 6/41
Updating : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64 7/41
Updating : firewalld-filesystem-0.6.3-8.el7_8.1.noarch 8/41
Updating : python-firewall-0.6.3-8.el7_8.1.noarch 9/41
Updating : kernel-tools-libs-3.10.0-1127.10.1.el7.x86_64 10/41
Updating : kernel-tools-3.10.0-1127.10.1.el7.x86_64 11/41
Updating : firewalld-0.6.3-8.el7_8.1.noarch 12/41
Updating : 7:lvm2-2.02.186-7.el7_8.2.x86_64 13/41
Updating : systemd-sysv-219-73.el7_8.6.x86_64 14/41
Updating : tzdata-2020a-1.el7.noarch 15/41
Updating : 32:bind-export-libs-9.11.4-16.P2.el7_8.6.x86_64 16/41
Updating : iprutils-2.4.17.1-3.el7_7.x86_64 17/41
Updating : binutils-2.27-43.base.el7_8.1.x86_64 18/41
Updating : yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch 19/41
Installing : kernel-3.10.0-1127.10.1.el7.x86_64 20/41
Updating : python-perf-3.10.0-1127.10.1.el7.x86_64 21/41
Cleanup : 7:lvm2-2.02.186-7.el7.x86_64 22/41
Cleanup : firewalld-0.6.3-8.el7.noarch 23/41
Cleanup : systemd-sysv-219-73.el7.1.x86_64 24/41
Cleanup : firewalld-filesystem-0.6.3-8.el7.noarch 25/41
Cleanup : python-firewall-0.6.3-8.el7.noarch 26/41
Cleanup : tzdata-2019c-1.el7.noarch 27/41
Cleanup : yum-plugin-fastestmirror-1.1.31-53.el7.noarch 28/41
Cleanup : 7:lvm2-libs-2.02.186-7.el7.x86_64 29/41
Cleanup : 7:device-mapper-event-1.02.164-7.el7.x86_64 30/41
Cleanup : 7:device-mapper-event-libs-1.02.164-7.el7.x86_64 31/41
Cleanup : 7:device-mapper-1.02.164-7.el7.x86_64 32/41
Cleanup : 7:device-mapper-libs-1.02.164-7.el7.x86_64 33/41
Cleanup : systemd-219-73.el7.1.x86_64 34/41
Cleanup : kernel-tools-3.10.0-1127.el7.x86_64 35/41
Cleanup : kernel-tools-libs-3.10.0-1127.el7.x86_64 36/41
Cleanup : systemd-libs-219-73.el7.1.x86_64 37/41
Cleanup : 32:bind-export-libs-9.11.4-16.P2.el7.x86_64 38/41
Cleanup : iprutils-2.4.17.1-3.el7.x86_64 39/41
Cleanup : binutils-2.27-43.base.el7.x86_64 40/41
Cleanup : python-perf-3.10.0-1127.el7.x86_64 41/41
Verifying : python-perf-3.10.0-1127.10.1.el7.x86_64 1/41
Verifying : kernel-3.10.0-1127.10.1.el7.x86_64 2/41
Verifying : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64 3/41
Verifying : 7:lvm2-2.02.186-7.el7_8.2.x86_64 4/41
Verifying : yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch 5/41
Verifying : 7:device-mapper-1.02.164-7.el7_8.2.x86_64 6/41
Verifying : systemd-sysv-219-73.el7_8.6.x86_64 7/41
Verifying : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64 8/41
Verifying : kernel-tools-libs-3.10.0-1127.10.1.el7.x86_64 9/41
Verifying : binutils-2.27-43.base.el7_8.1.x86_64 10/41
Verifying : systemd-219-73.el7_8.6.x86_64 11/41
Verifying : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64 12/41
Verifying : firewalld-0.6.3-8.el7_8.1.noarch 13/41
Verifying : iprutils-2.4.17.1-3.el7_7.x86_64 14/41
Verifying : systemd-libs-219-73.el7_8.6.x86_64 15/41
Verifying : kernel-tools-3.10.0-1127.10.1.el7.x86_64 16/41
Verifying : 32:bind-export-libs-9.11.4-16.P2.el7_8.6.x86_64 17/41
Verifying : tzdata-2020a-1.el7.noarch 18/41
Verifying : python-firewall-0.6.3-8.el7_8.1.noarch 19/41
Verifying : firewalld-filesystem-0.6.3-8.el7_8.1.noarch 20/41
Verifying : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64 21/41
Verifying : tzdata-2019c-1.el7.noarch 22/41
Verifying : firewalld-0.6.3-8.el7.noarch 23/41
Verifying : 7:device-mapper-event-1.02.164-7.el7.x86_64 24/41
Verifying : 32:bind-export-libs-9.11.4-16.P2.el7.x86_64 25/41
Verifying : 7:device-mapper-event-libs-1.02.164-7.el7.x86_64 26/41
Verifying : 7:lvm2-2.02.186-7.el7.x86_64 27/41
Verifying : systemd-219-73.el7.1.x86_64 28/41
Verifying : systemd-sysv-219-73.el7.1.x86_64 29/41
Verifying : firewalld-filesystem-0.6.3-8.el7.noarch 30/41
Verifying : systemd-libs-219-73.el7.1.x86_64 31/41
[main]
Verifying : python-firewall-0.6.3-8.el7.noarch 32/41
Verifying : 7:device-mapper-1.02.164-7.el7.x86_64 33/41
Verifying : 7:device-mapper-libs-1.02.164-7.el7.x86_64 34/41
Verifying : binutils-2.27-43.base.el7.x86_64 35/41
Verifying : kernel-tools-3.10.0-1127.el7.x86_64 36/41
Verifying : 7:lvm2-libs-2.02.186-7.el7.x86_64 37/41
Verifying : kernel-tools-libs-3.10.0-1127.el7.x86_64 38/41
Verifying : iprutils-2.4.17.1-3.el7.x86_64 39/41
Verifying : python-perf-3.10.0-1127.el7.x86_64 40/41
Verifying : yum-plugin-fastestmirror-1.1.31-53.el7.noarch 41/41 Installed:
kernel.x86_64 0:3.10.0-1127.10.1.el7
Updated:
bind-export-libs.x86_64 32:9.11.4-16.P2.el7_8.6 binutils.x86_64 0:2.27-43.base.el7_8.1
device-mapper.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event.x86_64 7:1.02.164-7.el7_8.2
device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2
firewalld.noarch 0:0.6.3-8.el7_8.1 firewalld-filesystem.noarch 0:0.6.3-8.el7_8.1
iprutils.x86_64 0:2.4.17.1-3.el7_7 kernel-tools.x86_64 0:3.10.0-1127.10.1.el7
kernel-tools-libs.x86_64 0:3.10.0-1127.10.1.el7 lvm2.x86_64 7:2.02.186-7.el7_8.2
lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 python-firewall.noarch 0:0.6.3-8.el7_8.1
python-perf.x86_64 0:3.10.0-1127.10.1.el7 systemd.x86_64 0:219-73.el7_8.6
systemd-libs.x86_64 0:219-73.el7_8.6 systemd-sysv.x86_64 0:219-73.el7_8.6
tzdata.noarch 0:2020a-1.el7 yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8
Complete!
[root@X11TESTSERVER ~]# reboot now
Connection to 23.111.168.58 closed by remote host.
Connection to 23.111.168.58 closed.
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ping 23.111.168.58
PING 23.111.168.58 (23.111.168.58) 56(84) bytes of data.
64 bytes from 23.111.168.58: icmp_seq=65 ttl=56 time=1.64 ms
64 bytes from 23.111.168.58: icmp_seq=66 ttl=56 time=1.72 ms
64 bytes from 23.111.168.58: icmp_seq=67 ttl=56 time=1.34 ms
64 bytes from 23.111.168.58: icmp_seq=68 ttl=56 time=1.31 ms
64 bytes from 23.111.168.58: icmp_seq=69 ttl=56 time=1.40 ms
64 bytes from 23.111.168.58: icmp_seq=70 ttl=56 time=1.36 ms
64 bytes from 23.111.168.58: icmp_seq=71 ttl=56 time=1.35 ms
64 bytes from 23.111.168.58: icmp_seq=72 ttl=56 time=1.36 ms
^C
--- 23.111.168.58 ping statistics ---
72 packets transmitted, 8 received, 88% packet loss, time 71072ms
rtt min/avg/max/mdev = 1.318/1.441/1.727/0.147 ms
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ssh root@23.111.168.58
root@23.111.168.58's password:
Last login: Sun Jun 14 23:07:53 2020 from 68-233-224-175.static.hvvc.us
[root@X11TESTSERVER ~]# nano /etc/yum.conf
-bash: nano: command not found
[root@X11TESTSERVER ~]# vi /etc/yum.conf
[root@X11TESTSERVER ~]# yum update -y
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.netdepot.com
* extras: mirror.teklinks.com
* updates: mirror.mojohost.com
base | 3.6 kB 00:00:00
extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
Resolving Dependencies
--> Running transaction check
---> Package microcode_ctl.x86_64 2:2.1-61.el7 will be updated
---> Package microcode_ctl.x86_64 2:2.1-61.6.el7_8 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================= Package Arch Version Repository Size
=========================================================================================================================Updating:
microcode_ctl x86_64 2:2.1-61.6.el7_8 updates 2.6 M
Transaction Summary
=========================================================================================================================Upgrade 1 Package
Total download size: 2.6 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
microcode_ctl-2.1-61.6.el7_8.x86_64.rpm | 2.6 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : 2:microcode_ctl-2.1-61.6.el7_8.x86_64 1/2
packet_write_wait: Connection to 23.111.168.58 port 22: Broken pipe
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ssh root@23.111.168.58
root@23.111.168.58's password:
Last login: Sun Jun 14 23:24:38 2020
[root@X11TESTSERVER ~]# rpm -qa | grep microcode_ctl
microcode_ctl-2.1-61.el7.x86_64
microcode_ctl-2.1-61.6.el7_8.x86_64
[root@X11TESTSERVER ~]#
|
|
|
|
Interesting things, I setup a new test and I yum updated without microcode_ctl, then proceeded to grab the update and installed it per @ManuelWolfshant recommendation, reboot and the server hangs at https://screencaps.donaldbennett.net/jp2launcher_KvC8sTt5Xg.png once you give it about 2-5 minutes the system will reboot and retry with no signs of pinging, happens on both kernels.
Using @robetek method of restoring access to the server works for me and subsequent shutdowns and boot-ups, are then successful.
1. Used grub editor during boot to include dis_ucode_ldr in boot arguments
2. Entered ctrl-x from grub editor to continue boot
3. yum remove microcode_ctl (note that this removed microcode_ctl 2.1-61.6.el7-8)
4. yum install microcode_ctl-2.1-61.el7
history2.txt (31,261 bytes)
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ssh root@23.111.168.58
The authenticity of host '23.111.168.58 (23.111.168.58)' can't be established.
ECDSA key fingerprint is SHA256:uFpvWWzR4wUIi9MjhWLAiidG7ILa3jxpnkPAbpa9CTY.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '23.111.168.58' (ECDSA) to the list of known hosts.
root@23.111.168.58's password:
Last failed login: Sun Jun 14 23:51:07 EDT 2020 from 159.65.13.233 on ssh:notty
There was 1 failed login attempt since the last successful login.
[root@X11TESTSERVER ~]# yum update --exclude=microcode_ctl -y
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Determining fastest mirrors
* base: bay.uchicago.edu
* extras: mirror.teklinks.com
* updates: mirror.mojohost.com
base | 3.6 kB 00:00:00
extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/4): updates/7/x86_64/primary_db | 2.1 MB 00:00:00
(2/4): base/7/x86_64/group_gz | 153 kB 00:00:00
(3/4): extras/7/x86_64/primary_db | 194 kB 00:00:00
(4/4): base/7/x86_64/primary_db | 6.1 MB 00:00:00
Resolving Dependencies
--> Running transaction check
---> Package bind-export-libs.x86_64 32:9.11.4-16.P2.el7 will be updated
---> Package bind-export-libs.x86_64 32:9.11.4-16.P2.el7_8.6 will be an update
---> Package binutils.x86_64 0:2.27-43.base.el7 will be updated
---> Package binutils.x86_64 0:2.27-43.base.el7_8.1 will be an update
---> Package device-mapper.x86_64 7:1.02.164-7.el7 will be updated
---> Package device-mapper.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-event.x86_64 7:1.02.164-7.el7 will be updated
---> Package device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-event-libs.x86_64 7:1.02.164-7.el7 will be updated
---> Package device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-libs.x86_64 7:1.02.164-7.el7 will be updated
---> Package device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package firewalld.noarch 0:0.6.3-8.el7 will be updated
---> Package firewalld.noarch 0:0.6.3-8.el7_8.1 will be an update
---> Package firewalld-filesystem.noarch 0:0.6.3-8.el7 will be updated
---> Package firewalld-filesystem.noarch 0:0.6.3-8.el7_8.1 will be an update
---> Package iprutils.x86_64 0:2.4.17.1-3.el7 will be updated
---> Package iprutils.x86_64 0:2.4.17.1-3.el7_7 will be an update
---> Package kernel.x86_64 0:3.10.0-1127.10.1.el7 will be installed
---> Package kernel-tools.x86_64 0:3.10.0-1127.el7 will be updated
---> Package kernel-tools.x86_64 0:3.10.0-1127.10.1.el7 will be an update
---> Package kernel-tools-libs.x86_64 0:3.10.0-1127.el7 will be updated
---> Package kernel-tools-libs.x86_64 0:3.10.0-1127.10.1.el7 will be an update
---> Package lvm2.x86_64 7:2.02.186-7.el7 will be updated
---> Package lvm2.x86_64 7:2.02.186-7.el7_8.2 will be an update
---> Package lvm2-libs.x86_64 7:2.02.186-7.el7 will be updated
---> Package lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 will be an update
---> Package python-firewall.noarch 0:0.6.3-8.el7 will be updated
---> Package python-firewall.noarch 0:0.6.3-8.el7_8.1 will be an update
---> Package python-perf.x86_64 0:3.10.0-1127.el7 will be updated
---> Package python-perf.x86_64 0:3.10.0-1127.10.1.el7 will be an update
---> Package systemd.x86_64 0:219-73.el7.1 will be updated
---> Package systemd.x86_64 0:219-73.el7_8.6 will be an update
---> Package systemd-libs.x86_64 0:219-73.el7.1 will be updated
---> Package systemd-libs.x86_64 0:219-73.el7_8.6 will be an update
---> Package systemd-sysv.x86_64 0:219-73.el7.1 will be updated
---> Package systemd-sysv.x86_64 0:219-73.el7_8.6 will be an update
---> Package tzdata.noarch 0:2019c-1.el7 will be updated
---> Package tzdata.noarch 0:2020a-1.el7 will be an update
---> Package yum-plugin-fastestmirror.noarch 0:1.1.31-53.el7 will be updated
---> Package yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================= Package Arch Version Repository Size
=========================================================================================================================Installing:
kernel x86_64 3.10.0-1127.10.1.el7 updates 50 M
Updating:
bind-export-libs x86_64 32:9.11.4-16.P2.el7_8.6 updates 1.1 M
binutils x86_64 2.27-43.base.el7_8.1 updates 5.9 M
device-mapper x86_64 7:1.02.164-7.el7_8.2 updates 295 k
device-mapper-event x86_64 7:1.02.164-7.el7_8.2 updates 191 k
device-mapper-event-libs x86_64 7:1.02.164-7.el7_8.2 updates 190 k
device-mapper-libs x86_64 7:1.02.164-7.el7_8.2 updates 324 k
firewalld noarch 0.6.3-8.el7_8.1 updates 443 k
firewalld-filesystem noarch 0.6.3-8.el7_8.1 updates 51 k
iprutils x86_64 2.4.17.1-3.el7_7 updates 243 k
kernel-tools x86_64 3.10.0-1127.10.1.el7 updates 8.0 M
kernel-tools-libs x86_64 3.10.0-1127.10.1.el7 updates 8.0 M
lvm2 x86_64 7:2.02.186-7.el7_8.2 updates 1.3 M
lvm2-libs x86_64 7:2.02.186-7.el7_8.2 updates 1.1 M
python-firewall noarch 0.6.3-8.el7_8.1 updates 354 k
python-perf x86_64 3.10.0-1127.10.1.el7 updates 8.0 M
systemd x86_64 219-73.el7_8.6 updates 5.1 M
systemd-libs x86_64 219-73.el7_8.6 updates 416 k
systemd-sysv x86_64 219-73.el7_8.6 updates 94 k
tzdata noarch 2020a-1.el7 updates 495 k
yum-plugin-fastestmirror noarch 1.1.31-54.el7_8 updates 34 k
Transaction Summary
=========================================================================================================================Install 1 Package
Upgrade 20 Packages
Total download size: 92 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
warning: /var/cache/yum/x86_64/7/updates/packages/bind-export-libs-9.11.4-16.P2.el7_8.6.x86_64.rpm: Header V3 RSA/SHA256
Signature, key ID f4a80eb5: NOKEY
Public key for bind-export-libs-9.11.4-16.P2.el7_8.6.x86_64.rpm is not installed
(1/21): bind-export-libs-9.11.4-16.P2.el7_8.6.x86_64.rpm | 1.1 MB 00:00:00
(2/21): device-mapper-1.02.164-7.el7_8.2.x86_64.rpm | 295 kB 00:00:00
(3/21): binutils-2.27-43.base.el7_8.1.x86_64.rpm | 5.9 MB 00:00:00
(4/21): device-mapper-event-1.02.164-7.el7_8.2.x86_64.rpm | 191 kB 00:00:00
(5/21): device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64.rpm | 190 kB 00:00:00
(6/21): device-mapper-libs-1.02.164-7.el7_8.2.x86_64.rpm | 324 kB 00:00:00
(7/21): firewalld-filesystem-0.6.3-8.el7_8.1.noarch.rpm | 51 kB 00:00:00
(8/21): firewalld-0.6.3-8.el7_8.1.noarch.rpm | 443 kB 00:00:00
(9/21): iprutils-2.4.17.1-3.el7_7.x86_64.rpm | 243 kB 00:00:00
(10/21): kernel-tools-3.10.0-1127.10.1.el7.x86_64.rpm | 8.0 MB 00:00:00
(11/21): kernel-3.10.0-1127.10.1.el7.x86_64.rpm | 50 MB 00:00:00
(12/21): kernel-tools-libs-3.10.0-1127.10.1.el7.x86_64.rpm | 8.0 MB 00:00:00
(13/21): lvm2-2.02.186-7.el7_8.2.x86_64.rpm | 1.3 MB 00:00:00
(14/21): lvm2-libs-2.02.186-7.el7_8.2.x86_64.rpm | 1.1 MB 00:00:00
(15/21): python-firewall-0.6.3-8.el7_8.1.noarch.rpm | 354 kB 00:00:00
(16/21): python-perf-3.10.0-1127.10.1.el7.x86_64.rpm | 8.0 MB 00:00:00
(17/21): systemd-219-73.el7_8.6.x86_64.rpm | 5.1 MB 00:00:00
(18/21): systemd-sysv-219-73.el7_8.6.x86_64.rpm | 94 kB 00:00:00
(19/21): systemd-libs-219-73.el7_8.6.x86_64.rpm | 416 kB 00:00:00
(20/21): yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch.rpm | 34 kB 00:00:00
(21/21): tzdata-2020a-1.el7.noarch.rpm | 495 kB 00:00:00
-------------------------------------------------------------------------------------------------------------------------Total 50 MB/s | 92 MB 00:00:01
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-8.2003.0.el7.centos.x86_64 (@anaconda)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : systemd-libs-219-73.el7_8.6.x86_64 1/41
Updating : systemd-219-73.el7_8.6.x86_64 2/41
Updating : 7:device-mapper-1.02.164-7.el7_8.2.x86_64 3/41
Updating : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64 4/41
Updating : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64 5/41
Updating : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64 6/41
Updating : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64 7/41
Updating : firewalld-filesystem-0.6.3-8.el7_8.1.noarch 8/41
Updating : python-firewall-0.6.3-8.el7_8.1.noarch 9/41
Updating : kernel-tools-libs-3.10.0-1127.10.1.el7.x86_64 10/41
Updating : kernel-tools-3.10.0-1127.10.1.el7.x86_64 11/41
Updating : firewalld-0.6.3-8.el7_8.1.noarch 12/41
Updating : 7:lvm2-2.02.186-7.el7_8.2.x86_64 13/41
Updating : systemd-sysv-219-73.el7_8.6.x86_64 14/41
Updating : tzdata-2020a-1.el7.noarch 15/41
Updating : 32:bind-export-libs-9.11.4-16.P2.el7_8.6.x86_64 16/41
Updating : iprutils-2.4.17.1-3.el7_7.x86_64 17/41
Updating : binutils-2.27-43.base.el7_8.1.x86_64 18/41
Updating : yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch 19/41
Installing : kernel-3.10.0-1127.10.1.el7.x86_64 20/41
Updating : python-perf-3.10.0-1127.10.1.el7.x86_64 21/41
Cleanup : 7:lvm2-2.02.186-7.el7.x86_64 22/41
Cleanup : firewalld-0.6.3-8.el7.noarch 23/41
Cleanup : systemd-sysv-219-73.el7.1.x86_64 24/41
Cleanup : firewalld-filesystem-0.6.3-8.el7.noarch 25/41
Cleanup : python-firewall-0.6.3-8.el7.noarch 26/41
Cleanup : tzdata-2019c-1.el7.noarch 27/41
Cleanup : yum-plugin-fastestmirror-1.1.31-53.el7.noarch 28/41
Cleanup : 7:lvm2-libs-2.02.186-7.el7.x86_64 29/41
Cleanup : 7:device-mapper-event-1.02.164-7.el7.x86_64 30/41
Cleanup : 7:device-mapper-event-libs-1.02.164-7.el7.x86_64 31/41
Cleanup : 7:device-mapper-1.02.164-7.el7.x86_64 32/41
Cleanup : 7:device-mapper-libs-1.02.164-7.el7.x86_64 33/41
Cleanup : systemd-219-73.el7.1.x86_64 34/41
Cleanup : kernel-tools-3.10.0-1127.el7.x86_64 35/41
Cleanup : kernel-tools-libs-3.10.0-1127.el7.x86_64 36/41
Cleanup : systemd-libs-219-73.el7.1.x86_64 37/41
Cleanup : 32:bind-export-libs-9.11.4-16.P2.el7.x86_64 38/41
Cleanup : iprutils-2.4.17.1-3.el7.x86_64 39/41
Cleanup : binutils-2.27-43.base.el7.x86_64 40/41
Cleanup : python-perf-3.10.0-1127.el7.x86_64 41/41
Verifying : python-perf-3.10.0-1127.10.1.el7.x86_64 1/41
Verifying : kernel-3.10.0-1127.10.1.el7.x86_64 2/41
Verifying : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64 3/41
Verifying : 7:lvm2-2.02.186-7.el7_8.2.x86_64 4/41
Verifying : yum-plugin-fastestmirror-1.1.31-54.el7_8.noarch 5/41
Verifying : 7:device-mapper-1.02.164-7.el7_8.2.x86_64 6/41
Verifying : systemd-sysv-219-73.el7_8.6.x86_64 7/41
Verifying : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64 8/41
Verifying : kernel-tools-libs-3.10.0-1127.10.1.el7.x86_64 9/41
Verifying : binutils-2.27-43.base.el7_8.1.x86_64 10/41
Verifying : systemd-219-73.el7_8.6.x86_64 11/41
Verifying : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64 12/41
Verifying : firewalld-0.6.3-8.el7_8.1.noarch 13/41
Verifying : iprutils-2.4.17.1-3.el7_7.x86_64 14/41
Verifying : systemd-libs-219-73.el7_8.6.x86_64 15/41
Verifying : kernel-tools-3.10.0-1127.10.1.el7.x86_64 16/41
Verifying : 32:bind-export-libs-9.11.4-16.P2.el7_8.6.x86_64 17/41
Verifying : tzdata-2020a-1.el7.noarch 18/41
Verifying : python-firewall-0.6.3-8.el7_8.1.noarch 19/41
Verifying : firewalld-filesystem-0.6.3-8.el7_8.1.noarch 20/41
Verifying : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64 21/41
Verifying : tzdata-2019c-1.el7.noarch 22/41
Verifying : firewalld-0.6.3-8.el7.noarch 23/41
Verifying : 7:device-mapper-event-1.02.164-7.el7.x86_64 24/41
Verifying : 32:bind-export-libs-9.11.4-16.P2.el7.x86_64 25/41
Verifying : 7:device-mapper-event-libs-1.02.164-7.el7.x86_64 26/41
Verifying : 7:lvm2-2.02.186-7.el7.x86_64 27/41
Verifying : systemd-219-73.el7.1.x86_64 28/41
Verifying : systemd-sysv-219-73.el7.1.x86_64 29/41
Verifying : firewalld-filesystem-0.6.3-8.el7.noarch 30/41
Verifying : systemd-libs-219-73.el7.1.x86_64 31/41
Verifying : python-firewall-0.6.3-8.el7.noarch 32/41
Verifying : 7:device-mapper-1.02.164-7.el7.x86_64 33/41
Verifying : 7:device-mapper-libs-1.02.164-7.el7.x86_64 34/41
Verifying : binutils-2.27-43.base.el7.x86_64 35/41
Verifying : kernel-tools-3.10.0-1127.el7.x86_64 36/41
Verifying : 7:lvm2-libs-2.02.186-7.el7.x86_64 37/41
Verifying : kernel-tools-libs-3.10.0-1127.el7.x86_64 38/41
Verifying : iprutils-2.4.17.1-3.el7.x86_64 39/41
Verifying : python-perf-3.10.0-1127.el7.x86_64 40/41
Verifying : yum-plugin-fastestmirror-1.1.31-53.el7.noarch 41/41
Installed:
kernel.x86_64 0:3.10.0-1127.10.1.el7
Updated:
bind-export-libs.x86_64 32:9.11.4-16.P2.el7_8.6 binutils.x86_64 0:2.27-43.base.el7_8.1
device-mapper.x86_64 7:1.02.164-7.el7_8.2 device-mapper-event.x86_64 7:1.02.164-7.el7_8.2
device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2
firewalld.noarch 0:0.6.3-8.el7_8.1 firewalld-filesystem.noarch 0:0.6.3-8.el7_8.1
iprutils.x86_64 0:2.4.17.1-3.el7_7 kernel-tools.x86_64 0:3.10.0-1127.10.1.el7
kernel-tools-libs.x86_64 0:3.10.0-1127.10.1.el7 lvm2.x86_64 7:2.02.186-7.el7_8.2
lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 python-firewall.noarch 0:0.6.3-8.el7_8.1
python-perf.x86_64 0:3.10.0-1127.10.1.el7 systemd.x86_64 0:219-73.el7_8.6
systemd-libs.x86_64 0:219-73.el7_8.6 systemd-sysv.x86_64 0:219-73.el7_8.6
tzdata.noarch 0:2020a-1.el7 yum-plugin-fastestmirror.noarch 0:1.1.31-54.el7_8
Complete!
[root@X11TESTSERVER ~]# wget http://ftp.usf.edu/pub/centos/7.8.2003/updates/x86_64/Packages/microcode_ctl-2.1-61.6.el7_8.x86_64.rpm
-bash: wget: command not found
[root@X11TESTSERVER ~]# yum install wget -y
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: bay.uchicago.edu
* extras: mirror.teklinks.com
* updates: mirror.mojohost.com
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-18.el7_6.1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================= Package Arch Version Repository Size
=========================================================================================================================Installing:
wget x86_64 1.14-18.el7_6.1 base 547 k
Transaction Summary
=========================================================================================================================Install 1 Package
Total download size: 547 k
Installed size: 2.0 M
Downloading packages:
wget-1.14-18.el7_6.1.x86_64.rpm | 547 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : wget-1.14-18.el7_6.1.x86_64 1/1
Verifying : wget-1.14-18.el7_6.1.x86_64 1/1
Installed:
wget.x86_64 0:1.14-18.el7_6.1
Complete!
[root@X11TESTSERVER ~]# wget http://ftp.usf.edu/pub/centos/7.8.2003/updates/x86_64/Packages/microcode_ctl-2.1-61.6.el7_8.x86_64.rpm
--2020-06-14 23:54:53-- http://ftp.usf.edu/pub/centos/7.8.2003/updates/x86_64/Packages/microcode_ctl-2.1-61.6.el7_8.x86_64.rpm
Resolving ftp.usf.edu (ftp.usf.edu)... 131.247.253.85, 2607:fe50:0:f30b::85
Connecting to ftp.usf.edu (ftp.usf.edu)|131.247.253.85|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2720752 (2.6M) [application/x-rpm]
Saving to: 'microcode_ctl-2.1-61.6.el7_8.x86_64.rpm'
100%[===============================================================================>] 2,720,752 6.19MB/s in 0.4s
2020-06-14 23:54:53 (6.19 MB/s) - 'microcode_ctl-2.1-61.6.el7_8.x86_64.rpm' saved [2720752/2720752]
[root@X11TESTSERVER ~]# rpm -U --noscripts microcode_ctl-2.1-61.6.el7_8.x86_64.rpm
[root@X11TESTSERVER ~]# reboot now
Connection to 23.111.168.58 closed by remote host.
Connection to 23.111.168.58 closed.
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ping 23.111.168.58
PING 23.111.168.58 (23.111.168.58) 56(84) bytes of data.
64 bytes from 23.111.168.58: icmp_seq=484 ttl=56 time=2.11 ms
64 bytes from 23.111.168.58: icmp_seq=485 ttl=56 time=1.58 ms
64 bytes from 23.111.168.58: icmp_seq=486 ttl=56 time=1.69 ms
^C
--- 23.111.168.58 ping statistics ---
486 packets transmitted, 3 received, 99% packet loss, time 485533ms
rtt min/avg/max/mdev = 1.584/1.796/2.112/0.232 ms
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ssh root@23.111.168.58
root@23.111.168.58's password:
Last failed login: Mon Jun 15 00:03:22 EDT 2020 from 80.ip-51-178-82.eu on ssh:notty
There were 2 failed login attempts since the last successful login.
Last login: Sun Jun 14 23:51:11 2020 from 68-233-224-175.static.hvvc.us
[root@X11TESTSERVER ~]# yum remove microcode_ctl
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Resolving Dependencies
--> Running transaction check
---> Package microcode_ctl.x86_64 2:2.1-61.6.el7_8 will be erased
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================= Package Arch Version Repository Size
=========================================================================================================================Removing:
microcode_ctl x86_64 2:2.1-61.6.el7_8 installed 3.2 M
Transaction Summary
=========================================================================================================================Remove 1 Package
Installed size: 3.2 M
Is this ok [y/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
Erasing : 2:microcode_ctl-2.1-61.6.el7_8.x86_64 1/1
Verifying : 2:microcode_ctl-2.1-61.6.el7_8.x86_64 1/1
Removed:
microcode_ctl.x86_64 2:2.1-61.6.el7_8
Complete!
[root@X11TESTSERVER ~]# yum install microcode_ctl-2.1-61.el7
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: bay.uchicago.edu
* extras: mirror.teklinks.com
* updates: mirror.mojohost.com
Resolving Dependencies
--> Running transaction check
---> Package microcode_ctl.x86_64 2:2.1-61.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
========================================================================================================================= Package Arch Version Repository Size
=========================================================================================================================Installing:
microcode_ctl x86_64 2:2.1-61.el7 base 2.5 M
Transaction Summary
=========================================================================================================================Install 1 Package
Total download size: 2.5 M
Installed size: 3.1 M
Is this ok [y/d/N]: y
Downloading packages:
microcode_ctl-2.1-61.el7.x86_64.rpm | 2.5 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : 2:microcode_ctl-2.1-61.el7.x86_64 1/1
Verifying : 2:microcode_ctl-2.1-61.el7.x86_64 1/1
Installed:
microcode_ctl.x86_64 2:2.1-61.el7
Complete!
[root@X11TESTSERVER ~]#
[root@X11TESTSERVER ~]# reboot now
Connection to 23.111.168.58 closed by remote host.
Connection to 23.111.168.58 closed.
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ping 23.111.168.58
PING 23.111.168.58 (23.111.168.58) 56(84) bytes of data.
64 bytes from 23.111.168.58: icmp_seq=74 ttl=56 time=1.68 ms
64 bytes from 23.111.168.58: icmp_seq=75 ttl=56 time=1.76 ms
^C
--- 23.111.168.58 ping statistics ---
75 packets transmitted, 2 received, 97% packet loss, time 74083ms
rtt min/avg/max/mdev = 1.688/1.724/1.760/0.036 ms
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ssh root@23.111.168.58
root@23.111.168.58's password:
Last failed login: Mon Jun 15 00:04:09 EDT 2020 from 106.13.203.171 on ssh:notty
There was 1 failed login attempt since the last successful login.
Last login: Mon Jun 15 00:03:28 2020 from 68-233-224-175.static.hvvc.us
[root@X11TESTSERVER ~]# rpm -qa | grep microcode_ctl
microcode_ctl-2.1-61.el7.x86_64
[root@X11TESTSERVER ~]# reboot now
Connection to 23.111.168.58 closed by remote host.
Connection to 23.111.168.58 closed.
dbennett@DonaldB-Work /mnt/c/Users/dbennett $ ping 23.111.168.58
PING 23.111.168.58 (23.111.168.58) 56(84) bytes of data.
64 bytes from 23.111.168.58: icmp_seq=71 ttl=56 time=1.93 ms
64 bytes from 23.111.168.58: icmp_seq=72 ttl=56 time=1.79 ms
64 bytes from 23.111.168.58: icmp_seq=73 ttl=56 time=1.62 ms
64 bytes from 23.111.168.58: icmp_seq=74 ttl=56 time=1.27 ms
^C
--- 23.111.168.58 ping statistics ---
74 packets transmitted, 4 received, 94% packet loss, time 73075ms
rtt min/avg/max/mdev = 1.277/1.659/1.939/0.249 ms
|
|
|
|
My server also hangs at https://screencaps.donaldbennett.net/jp2launcher_KvC8sTt5Xg.png
login have abrt
[root@soft1 /]# abrt-cli list --since 1592113010
id 839c14b7a66fda558af5f35505120a572883e911
reason: Kernel panic - not syncing: Timeout: Not all CPUs entered broadcast exception handler
time: 西元2020年06月15日 (週一) 08時56分31秒
uid: 0 (root)
count: 1
Directory: /var/spool/abrt/vmcore-127.0.0.1-2020-06-14-07:45:46
I don't know how to fix |
|
|
|
I followed your steps and cannot install microcode_ctl-2.1-61.el7
[root@soft1 tmp]# yum install microcode_ctl-2.1-61.el7
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: free.nchc.org.tw
* epel: mirrors.tuna.tsinghua.edu.cn
* extras: free.nchc.org.tw
* remi-php72: mirrors.tuna.tsinghua.edu.cn
* remi-safe: mirrors.tuna.tsinghua.edu.cn
* updates: free.nchc.org.tw
No package microcode_ctl-2.1-61.el7 available.
Error: Nothing to do |
|
|
|
Sorry, I found that I added exclude=microcode_ctl to yum.conf, so I cannot install |
|
|
|
It's normal now, and this problem remains
reason: Kernel panic - not syncing: Timeout: Not all CPUs entered broadcast exception handler |
|
|
|
I had this same issue yesterday on CentOS 7.7. Everything was going well, until the kernel panic. It said that it was going to reboot after 30 seconds. I waited at least several minutes, but my netbook doesn't display disk activity. I finally shut down the laptop since it appeared to be dead. I now have the symptoms described above, with duplicate packages. I need to figure out how to recover from this, or possibly use this as an opportunity to migrate to Stream 8.
Is there an issue with the microcode_ctl package? Will it only affect CentOS and not the production RHEL servers? |
|
|
|
Since CentOS is a rebuild of the source used to create RHEL I see no reason why the bug should not exist there too.
Intel published a new microcode tarball yesterday and it is noticeable that the 3 changed files in that vs the previous one, all are downgrades from version 0xdc to 0xd6 (now marked as "new") so this would appear to be an Intel bug not a Red Hat one. Presumably RH will pick that up in due course and there will be yet another microcode_ctl package released. |
|
|
|
relevant RHBZ is here:
https://bugzilla.redhat.com/show_bug.cgi?id=1846133
which refers to:
https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/issues/31 |
|
|
|
My Skylake-based systems were also affected. One was able to reboot by itself (I'm not sure why), while the other froze and required manual intervention (editing the Grub boot parameters). I see both systems now have two microcode_ctl packages installed, as reported by others:
microcode_ctl-2.1-61.el7
microcode_ctl-2.1-61.6.el7_8
When the the new microcode_ctl package is released and pushed to repos, will yum be able to clean up this problem? Or should I manually remove one or both of the already installed packages, as dbennet did above?
Thanks for any tips. |
|
|
|
You'll almost certainly need to clean up duplicate packages yourself and I suspect there will be more than just that one.
https://wiki.centos.org/HowTos/CleaningUpDuplicatePackages might help but don't forget the backups! |
|
|
|
aj_potc
If there are more than one duplicate packages, you should pay attention, because the duplicate packages configuration file will become 0byte, I am the victim, and I have updated php selinux bind... At the same time, these packages are duplicated, and the configuration file has become 0byte |
|
|
|
@nccsoft : https://access.redhat.com/solutions/3924551 |
|
|
|
That Red Hat solution is a great link, and it's what I used to resolve my issue. I almost posted it, but ultimately decided against it because it wasn't available to the general public.
The one thing it didn't do is get my GNOME desktop working again. I never get to the login prompt, so I have to press Ctrl-Alt F2, and log in at the command line. Oh, well, maybe this is a good excuse to ry CentOS Stream 8. Anyone have some ideas to get my desktop back? |
|
|
|
@__david__ : you could try removing the configuration files and/or reinstalling the DE. assuming the logs do not help, that is. |
|
|
|
Just an update on my attempt to clean up this mess on my two servers:
In my case, the only duplicate package on my systems was microcode_ctl. I attempted to use the link posted by @TrevorH, which suggests running the following command to remove duplicates:
package-cleanup --cleandupes --removenewestdupes
This appeared to do what it should by removing the latest, incompletely installed microcode_ctl version. Next, I did the following:
1. Rebooted the systems. One system rebooted fine, while the other hung. (No idea why -- they are identical.)
2. On the hung system, edited the Grub boot command to include dis_ucode_ldr in the boot arguments. System booted normally.
3. Ran "yum reinstall microcode_ctl-2.1-61.el7". (A regular "yum install" would not work here, as yum reported that the package was still installed. I though the package-cleanup would have resolved that, but apparently not.)
4. After a reboot, both systems now appear to boot properly.
5. Running "grep microcode /proc/cpuinfo" reveals that my systems are now running microcode revision 0xd6 -- for better or for worse.
At this point, I'm not sure whether I should block all future microcode_ctl updates, or pray that the future ones will be better tested. |
|
|
|
@aj_potc: if you are absolutely 1000% sure that there are no more bugs in your CPUs that would require new versions of microcode, blocking future updates is perfectly fine. As of praying... it depends to whom you pray. If you pray to Intel, it might be useful. If you pray to anyone else.... doubtful. |
|
|
|
I have not tried this, but if you are using lvm, you can make a snapshot. If the update goes bad, you can roll back the lvm snapshot. |
|
|
|
@ManuelWolfshant Thank you for the suggestion. I tried a yum groupremove "GNOME Desktop", and it removed only four files. I may try a yum reinstall, and see if that works. |
|
|
|
@__david__ : I am dead sure that the group contains way more than just 4 packages
LE: just tested "yum group install "Gnome Desktop" on a machine which had no DE installed:
Transaction Summary
===============================================================================================================================================================
Install 292 Packages (+756 Dependent packages)
|
|
|
|
@ManuelWolfshant In this usage, the word "pray" is a synonym for "hope," as that's all we can do.
At this point the only thing I can be sure of is that Intel's poorly-tested microcode mitigations have caused more harm than the bugs they're meant to fix. |
|
|
|
@ManuelWolfshant Thank you, I found the way you said on the Internet, but that can only help me delete duplicate packages, and cannot rescue my configuration file, because after installing microcode_ctl 2.1-61.6.el7-8, many of my configuration files have been Cleared to 0byte, but it is not important now, because I have restored it from my backup |
|
|
|
Latest update from Redhat: https://access.redhat.com/errata/RHEA-2020:2743
This looks to be resolved from Upstream, with RPM: microcode_ctl-2.1-61.10.el7_8.x86_64.rpm which replaces older RPM
https://screencaps.donaldbennett.net/jp2launcher_mb3eOYwA8o.png << Successful install with no reboot, running a reboot now after updates and OS boots just fine now with no issues or lock ups. |
|
|
|
Fixed by microcode_ctl-2.1-61.10.el7_8.x86_64.rpm |
|
