View Issue Details

IDProjectCategoryView StatusLast Update
0017749CentOS-7kernelpublic2020-09-23 20:25
Reportercmbrannon 
PrioritynormalSeveritycrashReproducibilityalways
Status newResolutionopen 
Product Version7.8-2003 
Target VersionFixed in Version 
Summary0017749: kernel-3.10.0-1127.19.1.el7.x86_64 crashes after an SSH connection attempt when running as a Xen PV guest on AMD Epyc Rome
DescriptionAs soon as I make an SSH connection to a CentOS 7 guest running under Xen PV, the guest crashes with the following output:

[174962.262346] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
[174962.262361] IP: [<ffffffff817886bc>] _raw_spin_lock+0xc/0x30
[174962.262373] PGD 0
[174962.262376] Oops: 0002 [#1] SMP
[174962.262381] Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ip_tables ext4 mbcache jbd2 xen_blkfront crct10dif_pclmul crct10dif_common xen_netfront crc32c_intel
[174962.262442] CPU: 0 PID: 11026 Comm: sshd Not tainted 3.10.0-1127.19.1.el7.x86_64 #1
[174962.262446] task: ffff88001daa1070 ti: ffff88000377c000 task.ti: ffff88000377c000
[174962.262451] RIP: e030:[<ffffffff817886bc>] [<ffffffff817886bc>] _raw_spin_lock+0xc/0x30
[174962.262457] RSP: e02b:ffff88000377fe10 EFLAGS: 00010046
[174962.262462] RAX: 0000000000000000 RBX: 00000000000000a0 RCX: ffffffff811582fe
[174962.262468] RDX: 0000000000000001 RSI: 0000000000000008 RDI: 0000000000000008
[174962.262475] RBP: ffff88000377fe38 R08: 000000000000003d R09: ffffffff8166efd3
[174962.262482] R10: ffff88001f81f100 R11: ffffea00000e3740 R12: ffff88001f810db0
[174962.262489] R13: 0006404000000000 R14: 0000000000000400 R15: 0000559a243c1ee0
[174962.262503] FS: 00007f6f3339e8c0(0000) GS:ffff88001f800000(0000) knlGS:0000000000000000
[174962.262511] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
[174962.262515] CR2: 0000000000000008 CR3: 0000000003a8c000 CR4: 0000000000040660
[174962.262523] Call Trace:
[174962.262531] [<ffffffff81036fa9>] ? speculation_ctrl_update+0xa9/0x1d0
[174962.262538] [<ffffffff8103766f>] speculation_ctrl_update_current+0x1f/0x30
[174962.262546] [<ffffffff81042349>] task_update_spec_tif+0x29/0x30
[174962.262553] [<ffffffff8104240d>] ssb_prctl_set+0xbd/0xf0
[174962.262559] [<ffffffff8104298e>] arch_seccomp_spec_mitigate+0x1e/0x20
[174962.262567] [<ffffffff81158098>] do_seccomp+0x518/0x810
[174962.262574] [<ffffffff81329fa4>] ? yama_task_prctl+0x24/0x110
[174962.262581] [<ffffffff811584b4>] prctl_set_seccomp+0x24/0x50
[174962.262589] [<ffffffff810ba24b>] SyS_prctl+0x3cb/0x4d0
[174962.262595] [<ffffffff81792e09>] ? system_call_after_swapgs+0x96/0x13a
[174962.262601] [<ffffffff81792e09>] ? system_call_after_swapgs+0x96/0x13a
[174962.262606] [<ffffffff81792e15>] ? system_call_after_swapgs+0xa2/0x13a
[174962.262612] [<ffffffff81792ed2>] system_call_fastpath+0x25/0x2a
[174962.262617] [<ffffffff81792e15>] ? system_call_after_swapgs+0xa2/0x13a
[174962.262622] Code: 5d c3 0f 1f 44 00 00 85 d2 74 e4 0f 1f 40 00 eb ed 66 0f 1f 44 00 00 b8 01 00 00 00 5d c3 90 66 66 66 66 90 31 c0 ba 01 00 00 00 <3e> 0f b1 17 85 c0 75 01 c3 55 89 c6 48 89 e5 e8 49 19 ff ff 5d
[174962.262672] RIP [<ffffffff817886bc>] _raw_spin_lock+0xc/0x30
[174962.262678] RSP <ffff88000377fe10>
[174962.262682] CR2: 0000000000000008
[174962.262689] ---[ end trace 36f5ee72136a0f8c ]---
[174962.262694] Kernel panic - not syncing: Fatal exception
Steps To Reproduce1. Boot a PV guest under the Xen hypervisor, on AMD Epyc Rome hardware.

2. Try to connect to the guest using SSH.

3. The guest crashes immediately and reliably.
Additional InformationI run Xen guests on various Intel processors, and recently, also on AMD Epyc Rome. The crash described in this report does not happen on any of the Intel processors, but it does happen on Epyc Rome. I have not seen it with kernels other than the CentOS 7 3.10.0-x kernels. I have only encountered it on Xen PV guests, not Xen HVM guests.

Adding the argument spec_store_bypass_disable=off prevents the crash, though this seems like an undesirable workaround.
Tags_unreleased_devel_pkgs
abrt_hash
URL

Activities

ManuelWolfshant

ManuelWolfshant

2020-09-23 20:25

manager   ~0037731

CentOS is a rebuild of the sources used to create RHEL and aims to reproduce RHEL bug for bug and feature for feature. Please submit your request to Redhat via bugzilla.redhat.com and if/when RH accepts it and incorporates it into RHEL and releases a patched version, then CentOS will pick it up automatically.
For easier tracking, please crosslink this bug with the one opened at bugzilla.redhat.com.

Issue History

Date Modified Username Field Change
2020-09-23 17:50 cmbrannon New Issue
2020-09-23 17:50 cmbrannon Tag Attached: _unreleased_devel_pkgs
2020-09-23 20:25 ManuelWolfshant Note Added: 0037731