View Issue Details

IDProjectCategoryView StatusLast Update
0014366CentOS-7kernelpublic2018-01-12 14:53
Reportersspans 
PrioritynormalSeveritycrashReproducibilityalways
Status newResolutionopen 
PlatformHPE ProLiant DL380 Gen9OS CentOS Linux OS Version 7.4.1708
Product Version 
Target VersionFixed in Version 
Summary0014366: KVM guest crashes when running on host with 3.10.0-693.11.6.el7.x86_64 kernel
Description
We are experiencing boot-time crashes on centos KVM guests after updating the host to a 7.4-meltdown kernel (3.10.0-693.11.6.el7.x86_64). Reproducing the issue seems to require a broadwell cpu (Xeon v4) and substantial cpu load.

Downgrading the kernel to either 3.10.0-693.2.2 or 3.10.0-693.5.2 seems to resolve the issue.
Steps To Reproduce1- Prepare a host with matching kernel and cpu:

> uname -r
3.10.0-693.11.6.el7.x86_64
sspans@mccppod025-hv12 [ ~ ] (15:13:37 - Fri Jan 12)
> grep -m1 model\ name /proc/cpuinfo
model name : Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz

2- Generate enough CPU load by running stress-ng on the host and in 3-4 centos vms:
stress-ng --cpu 16 --cpu-method matrixprod --metrics-brief --perf -t 3600

3- boot a centos 7 vm
Additional Information[ 0.046000] divide error: 0000 [#1] SMP
[ 0.046000] Modules linked in:
[ 0.046000] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W ------------ 3.10.0-693.11.6.el7.x86_64 #1
[ 0.046000] Hardware name: Mission Critical Cloud Cosmic KVM Hypervisor, BIOS 1.10.2-3.el7_4.1 04/01/2014
[ 0.046000] task: ffff88007bcd0000 ti: ffff88007bcd8000 task.ti: ffff88007bcd8000
[ 0.046000] RIP: 0010:[<ffffffff810f2304>] [<ffffffff810f2304>] clockevents_config.part.3+0x24/0xa0
[ 0.046000] RSP: 0000:ffff88007bcdbdc0 EFLAGS: 00010246
[ 0.046000] RAX: ffffffffffffffff RBX: ffff88007d011100 RCX: 0000000000000000
[ 0.046000] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffffffffff
[ 0.046000] RBP: ffff88007bcdbdd8 R08: 0000000000000001 R09: ffffffff81d9bd8a
[ 0.046000] R10: 00000000000000c1 R11: 00000000000000c0 R12: 000000000000e010
[ 0.046000] R13: 000000000000e018 R14: 0000000000000014 R15: 0000000000000013
[ 0.046000] FS: 0000000000000000(0000) GS:ffff88007d000000(0000) knlGS:0000000000000000
[ 0.046000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.046000] CR2: 00000000ffffffff CR3: 00000000019fa000 CR4: 00000000003606f0
[ 0.046000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 0.046000] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 0.046000] Call Trace:
[ 0.046000] [<ffffffff810f23a0>] clockevents_config_and_register+0x20/0x30
[ 0.046000] [<ffffffff8105341f>] setup_APIC_timer+0x10f/0x170
[ 0.046000] [<ffffffff81b65fee>] setup_boot_APIC_clock+0x4d7/0x4e3
[ 0.046000] [<ffffffff81b6403a>] native_smp_prepare_cpus+0x34f/0x3a0
[ 0.046000] [<ffffffff81b501bf>] kernel_init_freeable+0xbd/0x219
[ 0.046000] [<ffffffff81694d50>] ? rest_init+0x80/0x80
[ 0.046000] [<ffffffff81694d5e>] kernel_init+0xe/0xf0
[ 0.046000] [<ffffffff816b8798>] ret_from_fork+0x58/0x90
[ 0.046000] [<ffffffff81694d50>] ? rest_init+0x80/0x80
[ 0.046000] Code: c3 66 0f 1f 44 00 00 0f 1f 44 00 00 55 31 d2 89 f1 89 f6 41 b8 01 00 00 00 48 89 e5 41 55 41 54 53 48 89 fb 48 8b 7f 70 48 89 f8 <48> f7 f6 48 85 c0 74 0b 48 3d 58 02 00 00 41 89 c0 77 51 4c 8d
[ 0.046000] RIP [<ffffffff810f2304>] clockevents_config.part.3+0x24/0xa0
[ 0.046000] RSP <ffff88007bcdbdc0>
[ 0.047000] ---[ end trace 38e9cff91ea6c531 ]---
[ 0.048000] Kernel panic - not syncing: Fatal exception
Tagskernel, kvm
abrt_hash
URL

Activities

TrevorH

TrevorH

2018-01-12 14:26

developer   ~0030919

Because CentOS is a rebuild of RHEL, you'll need to raise this as a bug on bugzilla.redhat.com too and cross link the bugs. There won't be any fix for it coming from CentOS until it's fixed in RHEL.
sspans

sspans

2018-01-12 14:53

reporter   ~0030920

I've created https://bugzilla.redhat.com/show_bug.cgi?id=1533921

Issue History

Date Modified Username Field Change
2018-01-12 14:20 sspans New Issue
2018-01-12 14:20 sspans Tag Attached: kernel
2018-01-12 14:20 sspans Tag Attached: kvm
2018-01-12 14:26 TrevorH Note Added: 0030919
2018-01-12 14:53 sspans Note Added: 0030920