2018-01-23 17:22 UTC

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0014336CentOS-6kernelpublic2018-01-23 10:31
ReporterJeff_S 
PriorityurgentSeveritymajorReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformXen VMOSEL6OS Version6.9
Product Version6.9 
Target VersionFixed in Version 
Summary0014336: EL6 VM fails to boot after updating to kernel-2.6.32-696.18.7.el6.x86_64
DescriptionAfter updating a fairly stock 6.9 VM (on a Xen host) to the latest kernel package, the VM will not boot. It must hang early in the boot process because there are no logs written about the boot attempts, and I'm unable to get on the Rackspace console to see any details about where it's hanging.
Steps To ReproduceUpdate kernel, reboot.
Additional InformationServer boots fine when reverting to the previously installed kernel, kernel-2.6.32-696.13.2.el6.x86_64.
TagsNo tags attached.
Attached Files

-Relationships
related to 0014347acknowledgedJohnnyHughes CentOS-7 kernel-plus-3.10.0-693.11.6.el7.centos.plus under Xen4 runs in endless loop 
+Relationships

-Notes

~0030857

themiz (reporter)

I can confirm and reproduce the error.
Cannot even get to a console/grub to choose older kernel (so non logs, non error messages, nothing): I had to restore the server from snapshot.

Affected: Centos 6.9 PV hosts under xenserver 7.x with kernel 2.6.32-696.18.7.el6.x86_64
Not affected: Centos 7.4 HVM hosts (kernel 3.10.0-693.11.6.el7.x86_64)
(I haven't tried other combinations yet)

~0030858

themiz (reporter)

The topic is discussed in the forum:

https://www.centos.org/forums/viewtopic.php?f=13&t=65602

~0030861

dadapea (reporter)

xl dmesg shows:

(XEN) domain_crash_sync called from entry.S: fault at ffff82d080230983 create_bounce_frame+0x12b/0x13a
(XEN) Domain 32 (vcpu#0) crashed on cpu#20:
(XEN) ----[ Xen-4.6.6-8.el6 x86_64 debug=n Not tainted ]----
(XEN) CPU: 20
(XEN) RIP: e033:[<ffffffff812b7872>]
(XEN) RFLAGS: 0000000000000202 EM: 1 CONTEXT: pv guest (d32v0)
(XEN) rax: 0000000000000000 rbx: ffffffffff400000 rcx: 0000000000000004
(XEN) rdx: 0000000000000010 rsi: ffffffffff400000 rdi: ffffffff81a03e68
(XEN) rbp: ffffffff81a03e48 rsp: ffffffff81a03e00 r8: 0000000000000000
(XEN) r9: 0000000000007ff0 r10: 0000000000000000 r11: 0000000000000000
(XEN) r12: ffffffff81a03e58 r13: ffffffffff400000 r14: ffffffffff410000
(XEN) r15: ffffffff81a03e68 cr0: 000000008005003b cr4: 00000000000026e0
(XEN) cr3: 000000124628e000 cr2: ffffffffff400000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033
(XEN) Guest stack trace from rsp=ffffffff81a03e00:
(XEN) 0000000000000004 0000000000000000 0000000000000000 ffffffff812b7872
(XEN) 000000010000e030 0000000000010002 ffffffff81a03e48 000000000000e02b
(XEN) ffffffff812b7869 ffffffff81a03eb8 ffffffff81c80153 0000000000000000
(XEN) 0000000000000000 ffffffff81a03e98 ffffffff81c872a0 0000000000000000
(XEN) 6ece555bc906fdd1 ffffffffffffffff ffffffff81c872a0 0000000000000000
(XEN) ffffffff81a03f80 ffffffffffffffff 0000000000000000 ffffffff81a03f68
(XEN) ffffffff81c45605 0000000000000010 ffffffff81c872a0 0000000000000000
(XEN) 0000000000000000 ffffffffffffffff 0000000000000000 ffffffff81a03f08
(XEN) ffffffff8107e0ee ffffffff81a03f68 ffffffff8154b0cd 0000000000000010
(XEN) ffffffff81a03f78 ffffffff81a03f38 6ece555bc906fdd1 ffffffff81a03f58
(XEN) ffffffff81c872a0 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 0000000000000000 ffffffff81a03fa8 ffffffff81c3fdda 6ece555bc906fdd1
(XEN) ffffffff81c8a820 000000000204aa24 0000000000000000 0000000000000000
(XEN) 0000000000000000 ffffffff81a03fc8 ffffffff81c3f33a ffffffff81c34640
(XEN) ffffffff884f4000 ffffffff81a03ff8 ffffffff81c4309c 829822031f898975
(XEN) 000206c235200800 0000000000000000 0000000000000000 0000000000000000
(XEN) ffffffff864f1000 ffffffff864f2000 ffffffff864f3000 ffffffff864f4000
(XEN) ffffffff864f5000 ffffffff864f6000 ffffffff864f7000 ffffffff864f8000
(XEN) ffffffff864f9000 ffffffff864fa000 ffffffff864fb000 ffffffff864fc000
(XEN) ffffffff864fd000 ffffffff864fe000 ffffffff864ff000 ffffffff86500000
(XEN) d33v0: unhandled page fault (ec=0000)
(XEN) Pagetable walk from ffffffffff400000:
(XEN) L4[0x1ff] = 0000001246291067 0000000000001a91
(XEN) L3[0x1ff] = 0000001246292067 0000000000001a92
(XEN) L2[0x1fa] = 0000000000000000 ffffffffffffffff

~0030863

chrisdb (reporter)

I get a diffent type of crash:

(XEN) mm.c:2554:d238 Bad type (saw 7400000000000001 != exp 1000000000000000) for mfn 100abc7 (pfn 1e04)
(XEN) mm.c:986:d238 Attempt to create linear p.t. with write perms
(XEN) d238:v0: unhandled page fault (ec=0000)
(XEN) Pagetable walk from ffffffffff400000:
(XEN) L4[0x1ff] = 000000100af3a067 0000000000001a91
(XEN) L3[0x1ff] = 000000100af39067 0000000000001a92
(XEN) L2[0x1fa] = 0000000000000000 ffffffffffffffff
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 238 (vcpu#0) crashed on cpu#3:
(XEN) ----[ Xen-4.2.2 x86_64 debug=n Not tainted ]----
(XEN) CPU: 3
(XEN) RIP: e033:[<ffffffff812b7872>]
(XEN) RFLAGS: 0000000000000202 EM: 1 CONTEXT: pv guest
(XEN) rax: 0000000000000000 rbx: ffffffffff400000 rcx: 0000000000000004
(XEN) rdx: 0000000000000010 rsi: ffffffffff400000 rdi: ffffffff81a03e68
(XEN) rbp: ffffffff81a03e48 rsp: ffffffff81a03e00 r8: 0000000000000000
(XEN) r9: 0000000000007ff0 r10: 0000000000000000 r11: 0000000000000000
(XEN) r12: ffffffff81a03e58 r13: ffffffffff400000 r14: ffffffffff410000
(XEN) r15: ffffffff81a03e68 cr0: 000000008005003b cr4: 00000000000426f0
(XEN) cr3: 000000100af3d000 cr2: ffffffffff400000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033
(XEN) Guest stack trace from rsp=ffffffff81a03e00:
(XEN) 0000000000000004 0000000000000000 0000000000000000 ffffffff812b7872
(XEN) 000000010000e030 0000000000010002 ffffffff81a03e48 000000000000e02b
(XEN) ffffffff812b7869 ffffffff81a03eb8 ffffffff81c80153 0000000000000000
(XEN) 0000000000000000 ffffffff81a03e98 ffffffff81c872a0 0000000000000000
(XEN) ba1afdab0d64b832 ffffffffffffffff ffffffff81c872a0 0000000000000000
(XEN) ffffffff81a03f80 ffffffffffffffff 0000000000000000 ffffffff81a03f68
(XEN) ffffffff81c45605 0000000000000010 ffffffff81c872a0 0000000000000000
(XEN) 0000000000000000 ffffffffffffffff 0000000000000000 ffffffff81a03f08
(XEN) ffffffff8107e0ee ffffffff81a03f68 ffffffff8154b0cd 0000000000000010
(XEN) ffffffff81a03f78 ffffffff81a03f38 ba1afdab0d64b832 ffffffff81a03f58
(XEN) ffffffff81c872a0 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 0000000000000000 ffffffff81a03fa8 ffffffff81c3fdda ba1afdab0d64b832
(XEN) ffffffff81c8a820 000000000204aa24 0000000000000000 0000000000000000
(XEN) 0000000000000000 ffffffff81a03fc8 ffffffff81c3f33a ffffffff81c34640
(XEN) ffffffff86127000 ffffffff81a03ff8 ffffffff81c4309c 9f9822031f898975
(XEN) 000206d700200800 0000000000000000 0000000000000000 0000000000000000
(XEN) ffffffff85d24000 ffffffff85d25000 ffffffff85d26000 ffffffff85d27000
(XEN) ffffffff85d28000 ffffffff85d29000 ffffffff85d2a000 ffffffff85d2b000
(XEN) ffffffff85d2c000 ffffffff85d2d000 ffffffff85d2e000 ffffffff85d2f000
(XEN) ffffffff85d30000 ffffffff85d31000 ffffffff85d32000 ffffffff85d33000

~0030887

toracat (manager)

See also:

https://lists.centos.org/pipermail/centos-virt/2018-January/005712.html

~0030955

toracat (manager)

According to https://access.redhat.com/solutions/3312501 , Red Hat Engineering is currently working on this issue.
+Notes

-Issue History
Date Modified Username Field Change
2018-01-05 16:15 Jeff_S New Issue
2018-01-05 16:54 themiz Note Added: 0030857
2018-01-05 17:11 themiz Note Added: 0030858
2018-01-06 03:54 dadapea Note Added: 0030861
2018-01-06 15:10 chrisdb Note Added: 0030863
2018-01-09 07:01 toracat Relationship added related to 0014347
2018-01-09 07:01 toracat Status new => acknowledged
2018-01-09 07:08 toracat Note Added: 0030887
2018-01-17 12:41 toracat Note Added: 0030955
+Issue History