View Issue Details

IDProjectCategoryView StatusLast Update
0015233CentOS-7kernelpublic2018-09-05 00:52
Reporterhejiwen 
PrioritynormalSeverityblockReproducibilityunable to reproduce
Status newResolutionopen 
Product Version7.2.1511 
Target VersionFixed in Version 
Summary0015233: 3.10.0-327.el7.x86_64 kernel panic and crash under cluster environment
DescriptionWe have about 23 nodes with kernel 3.10.0-327.el7.x86_64 and these is a node cause kernel panic and crash.

the vmcore dmesg info as follows:

[2358201.454103] BUG: unable to handle kernel paging request at ffff887620eab71c
[2358201.454161] IP: [<ffffffff810c4aad>] find_busiest_group+0x14d/0x910
[2358201.454210] PGD 1f32067 PUD 0
[2358201.454236] Oops: 0000 [#1] SMP
[2358201.454262] Modules linked in: 8021q garp stp mrp llc tun ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ipmi_devintf rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_uverbs(OE) ib_umad(OE) mlx5_ib(OE) mlx5_core(OE) xfs libcrc32c mlx4_en(OE) vxlan ip6_udp_tunnel udp_tunnel mlx4_ib(OE) ib_sa(OE) ib_mad(OE) ib_core(OE) ib_addr(OE) mlx4_core(OE) mlx_compat(OE) ipmi_watchdog vfat fat iTCO_wdt iTCO_vendor_support mxm_wmi coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ses lpc_ich mei_me enclosure sb_edac mei sg pcspkr mfd_core i2c_i801 edac_core ipmi_ssif ipmi_si ipmi_msghandler shpchp wmi acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace sunrpc knem(OE) ip_tables ext4 mbcache jbd2
[2358201.454870] sd_mod crc_t10dif crct10dif_generic ast syscopyarea sysfillrect crct10dif_pclmul sysimgblt crct10dif_common drm_kms_helper crc32c_intel ttm ixgbe igb drm mdio ptp mpt3sas(OE) i2c_algo_bit pps_core raid_class i2c_core dca scsi_transport_sas
[2358201.455153] task: ffff881fe1652e00 ti: ffff881fe1668000 task.ti: ffff881fe1668000
[2358201.455194] RIP: 0010:[<ffffffff810c4aad>] [<ffffffff810c4aad>] find_busiest_group+0x14d/0x910
[2358201.455247] RSP: 0018:ffff881fe166b828 EFLAGS: 00010086
[2358201.455273] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: 0000000000000028
[2358201.455307] RDX: 0000000000000001 RSI: 00000000810c14bc RDI: 0000000000000001
[2358201.455343] RBP: ffff881fe166b828 R08: ffff881fe166b728 R09: 0000000000000000
[2358201.455378] R10: 0000000000000001 R11: ffff88102842c800 R12: ffff881fe166ba10
[2358201.455413] R13: ffff887620eab718 R14: ffff881fe166b870 R15: 0000000000000001
[2358201.455458] FS: 00007f7f7aee8700(0000) GS:ffff88103fc40000(0000) knlGS:0000000000000000
[2358201.455507] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[2358201.455536] CR2: ffff887620eab71c CR3: 0000000feffb5000 CR4: 00000000003407e0
[2358201.455571] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[2358201.455613] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[2358201.455655] Stack:
[2358201.455670] ffff881fe166b9a0 ffffffff810c4b10 ffff881028427800 0000000000000000
[2358201.455728] 000000010088551c 0000000000014780 0000000000014780 ffff881028427818
[2358201.455784] 0000000000000000 0000000000000000 0000000000000400 0000000000000400
[2358201.455840] Call Trace:
[2358201.455864] [<ffffffff810c4b10>] find_busiest_group+0x1b0/0x910
[2358201.455905] [<ffffffff813146d8>] ? swiotlb_map_sg_attrs+0x78/0x150
[2358201.455944] [<ffffffff810c5488>] load_balance+0x218/0x890
[2358201.455987] [<ffffffff810bb685>] ? sched_clock_cpu+0x85/0xc0
[2358201.456026] [<ffffffff810c609b>] idle_balance+0x14b/0x1e0
[2358201.456059] [<ffffffff811688d0>] ? sleep_on_page+0x20/0x20
[2358201.456097] [<ffffffff8163a77a>] __schedule+0x79a/0x900
[2358201.456128] [<ffffffff811688d0>] ? sleep_on_page+0x20/0x20
[2358201.456158] [<ffffffff8163a909>] schedule+0x29/0x70
[2358201.456187] [<ffffffff816385f9>] schedule_timeout+0x209/0x2d0
[2358201.456228] [<ffffffff812cc124>] ? blk_finish_plug+0x14/0x40
[2358201.456261] [<ffffffff81175cee>] ? __do_page_cache_readahead+0x1de/0x250
[2358201.457849] [<ffffffff8101c829>] ? read_tsc+0x9/0x10
[2358201.459446] [<ffffffff811688d0>] ? sleep_on_page+0x20/0x20
[2358201.461015] [<ffffffff81639f3e>] io_schedule_timeout+0xae/0x130
[2358201.462404] [<ffffffff81639fd8>] io_schedule+0x18/0x20
[2358201.463727] [<ffffffff811688de>] sleep_on_page_killable+0xe/0x40
[2358201.464867] [<ffffffff816388bb>] __wait_on_bit_lock+0x5b/0xc0
[2358201.465929] [<ffffffff81168a78>] __lock_page_killable+0x78/0xa0
[2358201.466955] [<ffffffff810a6b60>] ? wake_atomic_t_function+0x40/0x40
[2358201.468158] [<ffffffff8116acee>] generic_file_aio_read+0x50e/0x750
[2358201.469191] [<ffffffffa06cce41>] xfs_file_aio_read+0x151/0x2f0 [xfs]
[2358201.470174] [<ffffffff811ddcdd>] do_sync_read+0x8d/0xd0
[2358201.471117] [<ffffffff811de43c>] vfs_read+0x9c/0x170
[2358201.472029] [<ffffffff811df162>] SyS_pread64+0x92/0xc0
[2358201.472921] [<ffffffff81645909>] system_call_fastpath+0x16/0x1b
[2358201.473787] Code: 00 49 63 d7 4c 8b ad b8 fe ff ff 8b 85 cc fe ff ff 4c 03 2c d5 e0 b8 a5 81 48 89 95 a0 fe ff ff 44 89 ff 8b b5 c8 fe ff ff 85 c0 <41> 8b 4d 04 89 8d ac fe ff ff 74 77 e8 52 80 ff ff 8b 8d ac fe
[2358201.475562] RIP [<ffffffff810c4aad>] find_busiest_group+0x14d/0x910
[2358201.476413] RSP <ffff881fe166b828>
[2358201.477307] CR2: ffff887620eab71c


Any suggestions?
TagsNo tags attached.
abrt_hash
URL

Activities

tru

tru

2018-09-04 10:06

administrator   ~0032640

3.10.0-327.el7.x86_64 is no longer supported, please upgrade to the supported version.
hejiwen

hejiwen

2018-09-04 11:47

reporter   ~0032641

Thank you for the comment.
 which version should I upgrade,can you give me some advice? 514,693 or 862?
tigalch

tigalch

2018-09-04 12:14

manager   ~0032642

Only latest is supported - 'yum update' will bring your system up to date.
hejiwen

hejiwen

2018-09-05 00:52

reporter   ~0032645

Thank you for your advice。
The node can't visit internet, can I only upgrade kernel directly in centos7.2, does it have some compatible issues?

Issue History

Date Modified Username Field Change
2018-09-04 09:18 hejiwen New Issue
2018-09-04 10:06 tru Status new => feedback
2018-09-04 10:06 tru Note Added: 0032640
2018-09-04 11:47 hejiwen Note Added: 0032641
2018-09-04 11:47 hejiwen Status feedback => new
2018-09-04 12:14 tigalch Note Added: 0032642
2018-09-05 00:52 hejiwen Note Added: 0032645