View Issue Details

IDProjectCategoryView StatusLast Update
0017450CentOS-8blktracepublic2020-06-30 08:50
Reporteryinzg 
PriorityhighSeveritycrashReproducibilitysometimes
Status newResolutionopen 
Product Version8.0.1905 
Target VersionFixed in Version 
Summary0017450: bio_integrity_verify_fn+0x45/0x50 causes the kernel crash
DescriptionRecently , I am working on our nvme SSD in the kernel 4.18.0-80 CentOS 8 using fio test ,then the kernel crash, I get the crash info as the below,
[145069.300851] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
[145069.300855] PGD 0 P4D 0
[145069.300859] Oops: 0000 [#1] SMP PTI
[145069.300862] CPU: 21 PID: 21956 Comm: kworker/21:1H Kdump: loaded Tainted: G W --------- - - 4.18.0-80.el8.x86_64 #1
[145069.300863] Hardware name: Inspur NF5280M5/YZMB-00882-104, BIOS 4.1.13 01/16/2020
[145069.300871] Workqueue: kintegrityd bio_integrity_verify_fn
[145069.300874] RIP: 0010:bio_integrity_verify_fn+0x45/0x50
[145069.300876] Code: 00 48 85 c0 74 24 48 8b 50 08 48 8d 77 e8 48 89 df e8 4f fb ff ff 48 89 df 88 43 1a e8 f4 f9 ff ff 48 89 df 5b e9 1b 9a fc ff <48> 8b 00 eb d7 66 0f 1f 44 00 00 0f 1f 44 00 00 48 83 bf b0 00 00
[145069.300878] RSP: 0018:ffffb6afb2cc3e90 EFLAGS: 00010246
[145069.300880] RAX: 0000000000000000 RBX: ffff9fe015a2ab00 RCX: ffff9ff3df862d20
[145069.300882] RDX: 0000000000000001 RSI: ffff9ff3d08a7908 RDI: ffff9ff3d08a7900
[145069.300883] RBP: ffff9ff3df862d00 R08: 0000000000000008 R09: 0000000000647974
[145069.300885] R10: 8080808080808080 R11: 0000000000000010 R12: ffff9ff3df86b500
[145069.300886] R13: 0000000000000020 R14: ffff9ff2d733bb00 R15: ffff9ff3d08a7908
[145069.300888] FS: 0000000000000000(0000) GS:ffff9ff3df840000(0000) knlGS:0000000000000000
[145069.300890] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[145069.300891] CR2: 0000000000000000 CR3: 0000002cce20a002 CR4: 00000000007606e0
[145069.300893] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[145069.300894] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[145069.300895] PKRU: 55555554
[145069.300897] Call Trace:
[145069.300903] process_one_work+0x1a7/0x360
[145069.300906] worker_thread+0x30/0x390
[145069.300909] ? pwq_unbound_release_workfn+0xd0/0xd0
[145069.300911] kthread+0x112/0x130
[145069.300914] ? kthread_bind+0x30/0x30
[145069.300919] ret_from_fork+0x35/0x40
[145069.300921] Modules linked in: nvme uas usb_storage fuse xt_CHECKSUM ipt_MASQUERADE xt_conntrack ipt_REJECT nft_counter nf_nat_tftp nf_conntrack_tftp tun bridge stp llc nf_tables_set nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nft_chain_route_ipv6 nft_chain_nat_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat devlink nft_chain_route_ipv4 nf_conntrack ip6_tables ip_tables nft_compat ip_set nf_tables nfnetlink sunrpc vfat fat ext4 mbcache jbd2 intel_rapl skx_edac nfit libnvdimm x86_pkg_temp_thermal coretemp kvm_intel ipmi_ssif kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel iTCO_wdt iTCO_vendor_support intel_cstate joydev intel_uncore mei_me pcspkr
[145069.300963] sg nvme_core intel_rapl_perf i2c_i801 mei lpc_ich ioatdma ipmi_si dca ipmi_devintf ipmi_msghandler wmi acpi_power_meter xfs libcrc32c sd_mod ast i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm crc32c_intel libahci megaraid_sas libata dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nvme]
[145069.300985] CR2: 0000000000000000
Also Download the debuginfo (kernel-debuginfo-4.18.0-80.el8.x86_64/ kernel-debuginfo-common-x86_64-4.18.0-80.el8.x86_64 ), Using the crash tool in Centos 8 to analyse the vmcore, the following is the log:
 
crash> bt
PID: 913 TASK: ffff89e9173f1ec0 CPU: 23 COMMAND: "kworker/23:1H"
#0 [ffffa4838ed4bb90] machine_kexec at ffffffff94e535be
#1 [ffffa4838ed4bbe8] __crash_kexec at ffffffff94f4be3d
#2 [ffffa4838ed4bcb0] crash_kexec at ffffffff94f4d0dd
#3 [ffffa4838ed4bcc8] oops_end at ffffffff94e201cd
#4 [ffffa4838ed4bce8] no_context at ffffffff94e61f64
#5 [ffffa4838ed4bd40] __do_page_fault at ffffffff94e62c98
#6 [ffffa4838ed4bdb0] do_page_fault at ffffffff94e630f2
#7 [ffffa4838ed4bde0] page_fault at ffffffff9580114e
    [exception RIP: bio_integrity_verify_fn+69]
    RIP: ffffffff951efe35 RSP: ffffa4838ed4be90 RFLAGS: 00010246
    RAX: 0000000000000000 RBX: ffff89e7924eb200 RCX: ffff89e91f8e2d20
    RDX: 0000000000000001 RSI: ffff89e7590f8a08 RDI: ffff89e7590f8a00
    RBP: ffff89e91f8e2d00 R8: 0000000000000008 R9: 0000000000647974
    R10: 8080808080808080 R11: 0000000000000010 R12: ffff89e91f8eb500
    R13: 0000000000000020 R14: ffff89c1c0db0b40 R15: ffff89e7590f8a08
    ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#8 [ffffa4838ed4be98] process_one_work at ffffffff94ec6e97
#9 [ffffa4838ed4bed8] worker_thread at ffffffff94ec7380
#10 [ffffa4838ed4bf10] kthread at ffffffff94ecce52
#11 [ffffa4838ed4bf50] ret_from_fork at ffffffff95800255
crash> dis -l ffffffff951efe35
/usr/src/debug/kernel-4.18.0-80.el8/linux-4.18.0-80.el8.x86_64/./include/linux/blkdev.h: 1535
0xffffffff951efe35 <bio_integrity_verify_fn+69>: mov (%rax),%rax
crash>
Steps To Reproducerandom test using fio ,Until now there is no fixed rule to follow
TagsNo tags attached.

Activities

toracat

toracat

2020-06-15 01:17

manager   ~0037100

Only the latest version is supported. Can you update the kernel/system and see if the problem persists?
yinzg

yinzg

2020-06-15 01:28

reporter   ~0037101

Thanks! You mean the latest version is the latest CentOS 8.xxx or the latest kernel version 5.xxx ?
toracat

toracat

2020-06-15 07:47

manager   ~0037106

I mean the latest CentOS 8.xxx kernel.
yinzg

yinzg

2020-06-30 08:50

reporter   ~0037271

Hi,toracat, I use the latest CentOS-8.2.2004-x86_64(kernel:4.18.0-193.el8.x86_64) , however, there is the same question , Can you help confirm the problem why it happens? Thanks!

Issue History

Date Modified Username Field Change
2020-06-12 01:16 yinzg New Issue
2020-06-15 01:17 toracat Note Added: 0037100
2020-06-15 01:28 yinzg Note Added: 0037101
2020-06-15 07:47 toracat Note Added: 0037106
2020-06-30 08:50 yinzg Note Added: 0037271