View Issue Details

IDProjectCategoryView StatusLast Update
0013936CentOS-7hyperv-daemonspublic2017-12-07 16:10
Reporterjustabug 
PrioritynormalSeveritymajorReproducibilityalways
Status acknowledgedResolutionopen 
PlatformHyper-vOSCentos 7OS Versionlatest
Product Version 
Target VersionFixed in Version 
Summary0013936: centos system-udevd 100% CPU Hyper-v
DescriptionSame issue when update on different hyper-v hosts

centos system-udevd

root 507 78.9 0.7 48796 6732 ? Rs 20:11 13:28

/usr/lib/systemd/systemd-udevd


journalctl -b -u systemd-udevd.service

Sep 28 20:11:18 HOST1 systemd[1]: Starting udev Kernel Device Manager...
Sep 28 20:11:18 HOST1 systemd-udevd[244]: starting version 219
Sep 28 20:11:18 HOST1 systemd[1]: Started udev Kernel Device Manager.
Sep 28 20:11:20 HOST1 systemd[1]: Stopping udev Kernel Device Manager...
Sep 28 20:11:20 HOST1 systemd[1]: Stopped udev Kernel Device Manager.
Sep 28 20:11:21 HOST1 systemd[1]: Starting udev Kernel Device Manager...
Sep 28 20:11:21 HOST1 systemd-udevd[507]: starting version 219
Sep 28 20:11:21 HOST1 systemd[1]: Started udev Kernel Device Manager.
Steps To Reproduceyum update
Additional Informationis a Hyper-v on Windows 10 - same issue on 3 different servers

me just a ladmin htop mc ;)
TagsNo tags attached.
abrt_hash
URL

Activities

justabug

justabug

2017-10-04 18:27

reporter   ~0030301

udevadm monitor

UDEV [272.854060] remove /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.854937] remove /devices/LNXSYSTM:00/device:00/PNP0A03:00/device:08/VMBUS:01/2dd1ce17-079e-403c-b352-a1921ee207ee (vmbus)
avery seconds

maybe its same issue?
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1676635
justabug

justabug

2017-10-04 18:28

reporter   ~0030302

UDEV [272.854060] remove /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.854937] remove /devices/LNXSYSTM:00/device:00/PNP0A03:00/device:08/VMBUS:01/2dd1ce17-079e-403c-b352-a1921ee207ee (vmbus)
UDEV [272.854962] add /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.856394] add /devices/LNXSYSTM:00/device:00/PNP0A03:00/device:08/VMBUS:01/2dd1ce17-079e-403c-b352-a1921ee207ee (vmbus)
UDEV [272.856419] remove /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.857132] add /devices/virtual/ptp/ptp0 (ptp)
UDEV [272.857578] remove /devices/LNXSYSTM:00/device:00/PNP0A03:00/device:08/VMBUS:01/2dd1ce17-079e-403c-b352-a1921ee207ee (vmbus)
justabug

justabug

2017-10-04 18:39

reporter   ~0030303

Ubuntu dev solve this issue but not on Centos


https://github.com/coreos/bugs/issues/1036
tru

tru

2017-10-04 21:40

administrator   ~0030308

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1496927/comments/28 reads:
<cut>
I believe the following commit will fix this bug:

commit ed9ba608e4851144af8c7061cbb19f751c73e998
Author: Olaf Hering <email address hidden>
Date: Mon Dec 14 16:01:42 2015 -0800

    Drivers: hv: vss: run only on supported host versions
</cut>

So upstream has fixed it in 4.4.21, you need to ask/request for a backport on bugzilla.redhat.com first. OTOH, you can RFE the centosplus kernel.
toracat

toracat

2017-10-05 00:04

manager   ~0030309

Turns out commit ed9ba608e4851144af8c7061cbb19f751c73e998 is already in the EL7.4 kernel.
toracat

toracat

2017-10-05 00:09

manager   ~0030310

@justabug

What is the version of your kernel?
justabug

justabug

2017-10-05 00:15

reporter   ~0030311

was latest all updates installed, its happens with big updates like 600mb i think. Now I can't check because before update i do snapshot and restore all VM
bobnupp

bobnupp

2017-10-08 16:10

reporter   ~0030325

@toracat:

I reproduced the same issue as @justabug on two separate machines, each a Windows 10 laptop running CentOS 7 in Hyper-V. Here is the kernel version:

<file>
[bob@centos-bob ~]$ uname -r
3.10.0-693.2.2.el7.x86_64
[bob@centos-bob ~]$ cat /proc/version
Linux version 3.10.0-693.2.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Tue Sep 12 22:26:13 UTC 2017
</file>
toracat

toracat

2017-10-08 17:47

manager   ~0030327

@justabug and @bobnupp

Could you test-install ELRepo's kernel-ml and see if the latest mainline kernel has the problem you are seeing fixed?
bobnupp

bobnupp

2017-10-08 19:07

reporter   ~0030328

@toracat:

That worked. I installed the latest mainline kernel from ELRepo:
Linux version 4.13.5-1.el7.elrepo.x86_64 (mockbuild@Build64R7) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)) #1 SMP Thu Oct 5 08:24:09 EDT 2017

After rebooting into the new kernel, systemd-udevd is no longer pegging the CPU, and is no longer writing log entries after the normal startup sequence. (As a side note, the system start time is now much faster.)
tigalch

tigalch

2017-10-08 19:11

manager   ~0030329

With this findings you should raise the issue at RHs bugzilla. If/when the fix the issue in their kernel, CentOS will inherit the fix.
I'll leave toracat to comment on the possibilities of including this fix in the plus kernel (if the exact fix can be localized).
toracat

toracat

2017-10-09 00:22

manager   ~0030332

Sure, once the patch is identified, we could add it to the plus kernel.
justabug

justabug

2017-11-15 10:36

reporter   ~0030575

can i do safe yum update now ?
justabug

justabug

2017-12-07 15:47

reporter   ~0030723

rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
yum --enablerepo=elrepo-kernel install kernel-ml

uname -r
3.10.0-693.11.1.el7.x86_64

Linux version 3.10.0-693.11.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Mon Dec 4 23:52:40 UTC 2017
___
And still same issue


Package kernel-ml-4.14.4-1.el7.elrepo.x86_64 already installed and latest version
toracat

toracat

2017-12-07 16:10

manager   ~0030724

@justabug

By installing kernel-ml, you are testing the latest stable kernel from kernel.org. The CentOS kernel is built from the corresponding kernel from RHEL. The fact that kernel-ml works but not the CentOS kernel means the fix is present in the latest upstream (kernel.org) kernel but has not been backported to the CentOS (RHEL) kernel.

You need to file a bug report at http://bugzilla.redhat.com and ask that the issue be fixed as noted by @tigalch above.

Issue History

Date Modified Username Field Change
2017-09-28 17:40 justabug New Issue
2017-10-04 18:27 justabug Note Added: 0030301
2017-10-04 18:28 justabug Note Added: 0030302
2017-10-04 18:39 justabug Note Added: 0030303
2017-10-04 21:40 tru Note Added: 0030308
2017-10-05 00:04 toracat Note Added: 0030309
2017-10-05 00:09 toracat Note Added: 0030310
2017-10-05 00:09 toracat Status new => feedback
2017-10-05 00:15 justabug Note Added: 0030311
2017-10-05 00:15 justabug Status feedback => new
2017-10-08 16:10 bobnupp Note Added: 0030325
2017-10-08 17:47 toracat Note Added: 0030327
2017-10-08 19:07 bobnupp Note Added: 0030328
2017-10-08 19:11 tigalch Note Added: 0030329
2017-10-09 00:22 toracat Note Added: 0030332
2017-10-09 00:23 toracat Status new => acknowledged
2017-11-15 10:36 justabug Note Added: 0030575
2017-12-07 15:47 justabug Note Added: 0030723
2017-12-07 16:10 toracat Note Added: 0030724