View Issue Details

IDProjectCategoryView StatusLast Update
0015067CentOS-6kernelpublic2018-10-11 06:28
Reportersimix 
PrioritynormalSeveritycrashReproducibilityalways
Status resolvedResolutionfixed 
Product Version6.10 
Target VersionFixed in Version 
Summary0015067: Kernel panic with EL7 KVM guest on EL6 host on AMD server
DescriptionI have AMD servers running CentOS 6.10 as KVM host and on it guests
running CentOS 7.5.

Since the latest kernel update on CentOS 6.10,
kernel-2.6.32-754.2.1.el6.x86_64, I can't boot the CentOS 7.5 guests with
kernel-3.10.0-862.6.3.el7.x86_64 anymore.

Reverting CentOS 6.10 host to kernel-2.6.32-754.el6.x86_64 fixes the problem.

Screendump of the panic is attached, not very helpful I guess.

The CPU models where I could test it are:

cpu family : 15
model : 65
model name : Dual-Core AMD Opteron(tm) Processor 2218
stepping : 3

cpu family : 16
model : 6
model name : AMD Athlon(tm) II Neo N36L Dual-Core Processor
stepping : 3

Didn't test on newer AMD CPU family.

I'm wondering, does someone see the same issue? Does it not happen on
Intel CPUs?
Steps To ReproduceOn KVM host run CentOS 6.10 with latest kernel-2.6.32-754.2.1.el6.x86_64
On KVM guest run CentOS 7.5 with latest kernel-3.10.0-862.6.3.el7.x86_64

start CentOS 7.5 guest with "virsh start el7"

It crashes quite early in the boot process.
Tags2.6.32-754.2.1.el6.x86_64

Activities

simix

simix

2018-07-17 06:55

reporter  

el7-panic.png (14,391 bytes)
el7-panic.png (14,391 bytes)
simix

simix

2018-07-17 07:05

reporter   ~0032285

I've created this bug for CentOS 6.10 because the regression comes with the upgrade from kernel-2.6.32-754.el6.x86_64 to kernel-2.6.32-754.2.1.el6.x86_64.

However, only CentOS 7.5 crashes on the 6.10 KVM host. I've tested CentOS 6.10, CentOS 5.11 and FreeBSD 11.2 as guests and they are all happy.

So in the end it might also be a bug in CentOS 7.5 kernel-3.10.0-862.6.3.el7.x86_64.
toracat

toracat

2018-07-17 13:27

manager   ~0032287

I suggest you _reinstall_ the kernel and see if that fixes the issue.
simix

simix

2018-07-17 14:05

reporter   ~0032288

As written above I've verified the issue on two different servers with two different VM guests in two different locations. I'm wondering why you believe reinstalling would fix the issue? Am I missing something obvious?

Anyway I've done as you suggested, reinstalled the latest kernel in both the VM host and guest. It still crashes in the same way.
toracat

toracat

2018-07-17 15:58

manager   ~0032290

@simix

Sorry for the "noise". :(

I have now heard what looks like a similar case. The workaround in that case was to downgrade 2 packages on the host to:

qemu-img-0.12.1.2-2.506.el6_10.1.x86_64
qemu-kvm-0.12.1.2-2.506.el6_10.1.x86_64
toracat

toracat

2018-07-17 16:00

manager   ~0032291

Correction: those are the latest. You need to downgrade them.

qemu-img-0.12.1.2-2.503.el6_9.6.x86_64
qemu-kvm-0.12.1.2-2.503.el6_9.6.x86_64
simix

simix

2018-07-17 17:06

reporter   ~0032295

I can confirm that the following works:

kernel-2.6.32-754.el6.x86_64 & qemu-img/qemu-kvm0.12.1.2-2.506.el6_10.1.x86_64

or

kernel-2.6.32-754.2.1.el6.x86_64 & qemu-img/qemu-kvm-0.12.1.2-2.503.el6_9.6.x86_64

but this doesn't:

kernel-2.6.32-754.2.1.el6.x86_64 & qemu-img/qemu-kvm0.12.1.2-2.506.el6_10.1.x86_64
toracat

toracat

2018-07-17 17:19

manager   ~0032296

OK, this problem was confirmed with RHEL. So this is not a CentOS issue.
simix

simix

2018-07-18 06:07

reporter   ~0032297

Thanks, is there any more info available somewhere, like a bz# or something else?

BTW, I've just tested with kernel-3.10.0-862.9.1.el7.x86_64 on the CentOS
7 guest and it still crashes the same way.
toracat

toracat

2018-07-19 21:39

manager   ~0032321

I have no further info. Could you file a bug report upstream (Red Hat)? That seems to be the best way to collect more info.
anderson_thomas

anderson_thomas

2018-08-17 13:54

reporter   ~0032516

Hi,

same issue on kernel-22.6.32-754.3.5.el6.x86_64 & qemu-img/qemu-kvm0.12.1.2-2.506.el6_10.1.x86_64

Is there a solution to the problem?

Thank you!
anderson_thomas

anderson_thomas

2018-08-17 14:23

reporter   ~0032519

Hi,

sorry, Kernel is: kernel-2.6.32-754.3.5.el6.x86_64

Thank you!
anderson_thomas

anderson_thomas

2018-08-17 15:13

reporter   ~0032520

Hi,

This works:
=========

kernel-2.6.32-754.el6.x86_64 & qemu-kvm-0.12.1.2-2.503.el6_9.6.x86_64 AND qemu-img-0.12.1.2-2.503.el6_9.6.x86_64
kernel-2.6.32-754.3.5.el6.x86_64 & qemu-kvm-0.12.1.2-2.503.el6_9.6.x86_64 AND qemu-img-0.12.1.2-2.503.el6_9.6.x86_64

This doesn't work:
===============

kernel-2.6.32-754.3.5.el6.x86_64 & qemu-kvm-0.12.1.2-2.506.el6_10.1.x86_64 AND qemu-img-0.12.1.2-2.506.el6_10.1.x86_64


With latest CentOS 7.5 guests:
=========================

kernel-3.10.0-862.11.6.el7.x86_64


Thank you!
toracat

toracat

2018-09-09 04:59

manager   ~0032685

Upstream solution article"

https://access.redhat.com/solutions/3601721
"RHEL7 guest with AMD cpu fails to boot after upgrading underlying RHEL6 host with AMD cpu to kernel-2.6.32-754.3.5.el6.x86_64"
anderson_thomas

anderson_thomas

2018-09-09 09:44

reporter   ~0032686

Hi toracat,

is it possible to see the content here from https://access.redhat.com/solutions/3601721

Thank you!
toracat

toracat

2018-09-10 00:21

manager   ~0032688

@anderson_thomas

You need an active RH subscription to read the content. Basically what it says is that the issue is being tracked. A workaround is already presented in this bug tracker.
anderson_thomas

anderson_thomas

2018-10-10 03:07

reporter   ~0032890

Hi toracat,

Is a solution to this problem in sight, or is the workaround still the current solution?

Thank you!
toracat

toracat

2018-10-10 06:15

manager   ~0032892

@anderson_thomas

Good news. The latest kernel (2.6.32-754.6.3) that has just been released fixed the issue.
toracat

toracat

2018-10-10 06:32

manager   ~0032893

From the changelog:

- [x86] Add host_initiated check in reading MSR_AMD64_VIRT_SPEC_CTRL (Wei Huang) [1608576]
- [x86] KVM: pass host_initiated to functions that read MSRs (Wei Huang) [1608576]
anderson_thomas

anderson_thomas

2018-10-10 07:53

reporter   ~0032895

Hi toracat,

thank you so much for posting, that the issue was solved!

Thnak you!
simix

simix

2018-10-11 04:51

reporter   ~0032908

I can confirm that the latest kernel-2.6.32-754.6.3.el6 fixes the issue.

Thanks!
toracat

toracat

2018-10-11 06:28

manager   ~0032910

Thanks for the feedback. Closing as 'resolved'.

Issue History

Date Modified Username Field Change
2018-07-17 06:55 simix New Issue
2018-07-17 06:55 simix File Added: el7-panic.png
2018-07-17 06:55 simix Tag Attached: 2.6.32-754.2.1.el6.x86_64
2018-07-17 07:05 simix Note Added: 0032285
2018-07-17 13:27 toracat Note Added: 0032287
2018-07-17 14:05 simix Note Added: 0032288
2018-07-17 15:58 toracat Note Added: 0032290
2018-07-17 16:00 toracat Note Added: 0032291
2018-07-17 17:06 simix Note Added: 0032295
2018-07-17 17:19 toracat Note Added: 0032296
2018-07-18 06:07 simix Note Added: 0032297
2018-07-19 21:39 toracat Note Added: 0032321
2018-08-17 13:54 anderson_thomas Note Added: 0032516
2018-08-17 14:23 anderson_thomas Note Added: 0032519
2018-08-17 15:13 anderson_thomas Note Added: 0032520
2018-08-18 11:33 isvelov Tag Attached: "Network"
2018-08-18 11:33 isvelov Tag Detached: "Network"
2018-09-09 04:59 toracat Note Added: 0032685
2018-09-09 09:44 anderson_thomas Note Added: 0032686
2018-09-10 00:21 toracat Note Added: 0032688
2018-09-10 00:22 toracat Status new => acknowledged
2018-10-10 03:07 anderson_thomas Note Added: 0032890
2018-10-10 06:15 toracat Note Added: 0032892
2018-10-10 06:32 toracat Note Added: 0032893
2018-10-10 07:53 anderson_thomas Note Added: 0032895
2018-10-11 04:51 simix Note Added: 0032908
2018-10-11 06:28 toracat Status acknowledged => resolved
2018-10-11 06:28 toracat Resolution open => fixed
2018-10-11 06:28 toracat Note Added: 0032910