CentOS Bug Tracker - CentOS-7
View Issue Details
0013936CentOS-7hyperv-daemonspublic2017-09-28 17:402017-12-07 16:10
Reporterjustabug 
PrioritynormalSeveritymajorReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformHyper-vOSCentos 7OS Versionlatest
Product Version 
Target VersionFixed in Version 
abrt_hash
URL
Summary0013936: centos system-udevd 100% CPU Hyper-v
DescriptionSame issue when update on different hyper-v hosts

centos system-udevd

root 507 78.9 0.7 48796 6732 ? Rs 20:11 13:28

/usr/lib/systemd/systemd-udevd


journalctl -b -u systemd-udevd.service

Sep 28 20:11:18 HOST1 systemd[1]: Starting udev Kernel Device Manager...
Sep 28 20:11:18 HOST1 systemd-udevd[244]: starting version 219
Sep 28 20:11:18 HOST1 systemd[1]: Started udev Kernel Device Manager.
Sep 28 20:11:20 HOST1 systemd[1]: Stopping udev Kernel Device Manager...
Sep 28 20:11:20 HOST1 systemd[1]: Stopped udev Kernel Device Manager.
Sep 28 20:11:21 HOST1 systemd[1]: Starting udev Kernel Device Manager...
Sep 28 20:11:21 HOST1 systemd-udevd[507]: starting version 219
Sep 28 20:11:21 HOST1 systemd[1]: Started udev Kernel Device Manager.
Steps To Reproduceyum update
Additional Informationis a Hyper-v on Windows 10 - same issue on 3 different servers

me just a ladmin htop mc ;)
TagsNo tags attached.
Attached Files

Notes
(0030301)
justabug   
2017-10-04 18:27   
udevadm monitor

UDEV [272.854060] remove /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.854937] remove /devices/LNXSYSTM:00/device:00/PNP0A03:00/device:08/VMBUS:01/2dd1ce17-079e-403c-b352-a1921ee207ee (vmbus)
avery seconds

maybe its same issue?
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1676635
(0030302)
justabug   
2017-10-04 18:28   
UDEV [272.854060] remove /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.854937] remove /devices/LNXSYSTM:00/device:00/PNP0A03:00/device:08/VMBUS:01/2dd1ce17-079e-403c-b352-a1921ee207ee (vmbus)
UDEV [272.854962] add /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.856394] add /devices/LNXSYSTM:00/device:00/PNP0A03:00/device:08/VMBUS:01/2dd1ce17-079e-403c-b352-a1921ee207ee (vmbus)
UDEV [272.856419] remove /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.857132] add /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.857578] remove /devices/LNXSYSTM:00/device:00/PNP0A03:00/device:08/VMBUS:01/2dd1ce17-079e-403c-b352-a1921ee207ee (vmbus)
(0030303)
justabug   
2017-10-04 18:39   
Ubuntu dev solve this issue but not on Centos


https://github.com/coreos/bugs/issues/1036
(0030308)
tru   
2017-10-04 21:40   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1496927/comments/28 reads:
<cut>
I believe the following commit will fix this bug:

commit ed9ba608e4851144af8c7061cbb19f751c73e998
Author: Olaf Hering <email address hidden>
Date: Mon Dec 14 16:01:42 2015 -0800

    Drivers: hv: vss: run only on supported host versions
</cut>

So upstream has fixed it in 4.4.21, you need to ask/request for a backport on bugzilla.redhat.com first. OTOH, you can RFE the centosplus kernel.
(0030309)
toracat   
2017-10-05 00:04   
Turns out commit ed9ba608e4851144af8c7061cbb19f751c73e998 is already in the EL7.4 kernel.
(0030310)
toracat   
2017-10-05 00:09   
@justabug

What is the version of your kernel?
(0030311)
justabug   
2017-10-05 00:15   
was latest all updates installed, its happens with big updates like 600mb i think. Now I can't check because before update i do snapshot and restore all VM
(0030325)
bobnupp   
2017-10-08 16:10   
@toracat:

I reproduced the same issue as @justabug on two separate machines, each a Windows 10 laptop running CentOS 7 in Hyper-V. Here is the kernel version:

<file>
[bob@centos-bob ~]$ uname -r
3.10.0-693.2.2.el7.x86_64
[bob@centos-bob ~]$ cat /proc/version
Linux version 3.10.0-693.2.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Tue Sep 12 22:26:13 UTC 2017
</file>
(0030327)
toracat   
2017-10-08 17:47   
@justabug and @bobnupp

Could you test-install ELRepo's kernel-ml and see if the latest mainline kernel has the problem you are seeing fixed?
(0030328)
bobnupp   
2017-10-08 19:07   
@toracat:

That worked. I installed the latest mainline kernel from ELRepo:
Linux version 4.13.5-1.el7.elrepo.x86_64 (mockbuild@Build64R7) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)) #1 SMP Thu Oct 5 08:24:09 EDT 2017

After rebooting into the new kernel, systemd-udevd is no longer pegging the CPU, and is no longer writing log entries after the normal startup sequence. (As a side note, the system start time is now much faster.)
(0030329)
tigalch   
2017-10-08 19:11   
With this findings you should raise the issue at RHs bugzilla. If/when the fix the issue in their kernel, CentOS will inherit the fix.
I'll leave toracat to comment on the possibilities of including this fix in the plus kernel (if the exact fix can be localized).
(0030332)
toracat   
2017-10-09 00:22   
Sure, once the patch is identified, we could add it to the plus kernel.
(0030575)
justabug   
2017-11-15 10:36   
can i do safe yum update now ?
(0030723)
justabug   
2017-12-07 15:47   
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
yum --enablerepo=elrepo-kernel install kernel-ml

uname -r
3.10.0-693.11.1.el7.x86_64

Linux version 3.10.0-693.11.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Mon Dec 4 23:52:40 UTC 2017
___
And still same issue


Package kernel-ml-4.14.4-1.el7.elrepo.x86_64 already installed and latest version
(0030724)
toracat   
2017-12-07 16:10   
@justabug

By installing kernel-ml, you are testing the latest stable kernel from kernel.org. The CentOS kernel is built from the corresponding kernel from RHEL. The fact that kernel-ml works but not the CentOS kernel means the fix is present in the latest upstream (kernel.org) kernel but has not been backported to the CentOS (RHEL) kernel.

You need to file a bug report at http://bugzilla.redhat.com and ask that the issue be fixed as noted by @tigalch above.

Issue History
2017-09-28 17:40justabugNew Issue
2017-10-04 18:27justabugNote Added: 0030301
2017-10-04 18:28justabugNote Added: 0030302
2017-10-04 18:39justabugNote Added: 0030303
2017-10-04 21:40truNote Added: 0030308
2017-10-05 00:04toracatNote Added: 0030309
2017-10-05 00:09toracatNote Added: 0030310
2017-10-05 00:09toracatStatusnew => feedback
2017-10-05 00:15justabugNote Added: 0030311
2017-10-05 00:15justabugStatusfeedback => new
2017-10-08 16:10bobnuppNote Added: 0030325
2017-10-08 17:47toracatNote Added: 0030327
2017-10-08 19:07bobnuppNote Added: 0030328
2017-10-08 19:11tigalchNote Added: 0030329
2017-10-09 00:22toracatNote Added: 0030332
2017-10-09 00:23toracatStatusnew => acknowledged
2017-11-15 10:36justabugNote Added: 0030575
2017-12-07 15:47justabugNote Added: 0030723
2017-12-07 16:10toracatNote Added: 0030724