View Issue Details

IDProjectCategoryView StatusLast Update
0015187CentOS-7kernelpublic2018-08-17 07:28
Reporterhtzk 
PriorityimmediateSeveritycrashReproducibilitysometimes
Status newResolutionopen 
Product Version7.4.1708 
Target VersionFixed in Version 
Summary0015187: BUG kmalloc-256(3380:526a2ac43ace8f2ea1cd3d35611ec14a915a102b9f4fe2094e4453d2e128731a)
Description[310365.325308] BUG kmalloc-256(3380:526a2ac43ace8f2ea1cd3d35611ec14a915a102b9f4fe2094e4453d2e128731a) (Tainted: G B W ------------ ): Objects remaining in kmalloc-256(3380:526a2ac43ace8f2ea1cd3d35611ec14a915a102b9f4fe2094e4453d2e1287
[310365.325868] -----------------------------------------------------------------------------

[310365.326524] INFO: Slab 0xffffea000cecb500 objects=64 used=62 fp=0xffff88033b2d4d00 flags=0x2fffff00004080
[310365.326883] CPU: 15 PID: 169758 Comm: dockerd Tainted: G B W ------------ 3.10.0-693.5.2.el7.x86_64 #1
[310365.326886] Hardware name: Dell Inc. PowerEdge R430/0CN7X8, BIOS 2.4.2 01/09/2017
[310365.326890] ffffea000cecb500 00000000d85b2edc ffff88091cedb9c0 ffffffff816a3e51
[310365.326895] ffff88091cedba98 ffffffff811dbec4 ffff881d00000020 ffff88091cedbaa8
[310365.326899] ffff88091cedba58 656a624f81910018 616d657220737463 6e6920676e696e69
[310365.326903] Call Trace:
[310365.326914] [<ffffffff816a3e51>] dump_stack+0x19/0x1b
[310365.326922] [<ffffffff811dbec4>] slab_err+0xb4/0xe0
[310365.326927] [<ffffffff811e0593>] ? __kmalloc+0x1e3/0x230
[310365.326933] [<ffffffff811e1887>] ? kmem_cache_close+0x127/0x2e0
[310365.326937] [<ffffffff811e18a9>] kmem_cache_close+0x149/0x2e0
[310365.326941] [<ffffffff811e1a54>] __kmem_cache_shutdown+0x14/0x80
[310365.326948] [<ffffffff811a67e4>] kmem_cache_destroy+0x44/0xf0
[310365.326953] [<ffffffff811f5f89>] kmem_cache_destroy_memcg_children+0x89/0xb0
[310365.326957] [<ffffffff811a67b9>] kmem_cache_destroy+0x19/0xf0
[310365.326965] [<ffffffff8123b0fe>] bioset_free+0xce/0x110
[310365.326998] [<ffffffffc000ae7e>] dm_io_client_destroy+0x1e/0x30 [dm_mod]
[310365.327007] [<ffffffffc000bff5>] dm_kcopyd_client_destroy+0xc5/0x110 [dm_mod]
[310365.327011] [<ffffffff811dee33>] ? kfree+0x103/0x140
[310365.327017] [<ffffffffc04a3b2f>] __pool_dec+0x7f/0x100 [dm_thin_pool]
[310365.327022] [<ffffffffc04a3bde>] pool_dtr+0x2e/0x70 [dm_thin_pool]
[310365.327031] [<ffffffffc00056c0>] dm_table_destroy+0x70/0x120 [dm_mod]
[310365.327041] [<ffffffffc00016b4>] __dm_destroy+0x144/0x310 [dm_mod]
[310365.327049] [<ffffffffc0003b43>] dm_destroy+0x13/0x20 [dm_mod]
[310365.327057] [<ffffffffc00099be>] dev_remove+0x11e/0x180 [dm_mod]
[310365.327064] [<ffffffffc00098a0>] ? dev_suspend+0x250/0x250 [dm_mod]
[310365.327072] [<ffffffffc000a015>] ctl_ioctl+0x1e5/0x500 [dm_mod]
[310365.327080] [<ffffffffc000a343>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[310365.327086] [<ffffffff812151bd>] do_vfs_ioctl+0x33d/0x540
[310365.327091] [<ffffffff816b0091>] ? __do_page_fault+0x171/0x450
[310365.327094] [<ffffffff81215461>] SyS_ioctl+0xa1/0xc0
[310365.327098] [<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
[310365.327102] INFO: Object 0xffff88033b2d4000 @offset=0
[310365.327764] INFO: Object 0xffff88033b2d4100 @offset=256
[310365.328206] INFO: Object 0xffff88033b2d4200 @offset=512
[310365.328560] INFO: Object 0xffff88033b2d4300 @offset=768
[310365.328964] INFO: Object 0xffff88033b2d4400 @offset=1024
[310365.329369] INFO: Object 0xffff88033b2d4500 @offset=1280
[310365.329736] INFO: Object 0xffff88033b2d4600 @offset=1536
[310365.330125] INFO: Object 0xffff88033b2d4700 @offset=1792
[310365.330481] INFO: Object 0xffff88033b2d4800 @offset=2048
[310365.330882] INFO: Object 0xffff88033b2d4900 @offset=2304
[310365.331286] INFO: Object 0xffff88033b2d4a00 @offset=2560
[310365.331653] INFO: Object 0xffff88033b2d4b00 @offset=2816
[310365.332043] INFO: Object 0xffff88033b2d4c00 @offset=3072
[310365.332399] INFO: Object 0xffff88033b2d4e00 @offset=3584
[310365.332765] INFO: Object 0xffff88033b2d4f00 @offset=3840
[310365.333168] INFO: Object 0xffff88033b2d5000 @offset=4096
[310365.333524] INFO: Object 0xffff88033b2d5100 @offset=4352
[310365.333924] INFO: Object 0xffff88033b2d5200 @offset=4608
[310365.334326] INFO: Object 0xffff88033b2d5300 @offset=4864
[310365.334690] INFO: Object 0xffff88033b2d5400 @offset=5120
[310365.335080] INFO: Object 0xffff88033b2d5500 @offset=5376
[310365.335435] INFO: Object 0xffff88033b2d5600 @offset=5632
[310365.335800] INFO: Object 0xffff88033b2d5700 @offset=5888
[310365.336202] INFO: Object 0xffff88033b2d5800 @offset=6144
[310365.336558] INFO: Object 0xffff88033b2d5900 @offset=6400
[310365.336923] INFO: Object 0xffff88033b2d5a00 @offset=6656
[310365.337326] INFO: Object 0xffff88033b2d5b00 @offset=6912
[310365.337690] INFO: Object 0xffff88033b2d5c00 @offset=7168
[310365.338080] INFO: Object 0xffff88033b2d5d00 @offset=7424
[310365.338435] INFO: Object 0xffff88033b2d5e00 @offset=7680
[310365.338801] INFO: Object 0xffff88033b2d5f00 @offset=7936
[310365.339201] INFO: Object 0xffff88033b2d6000 @offset=8192
[310365.339556] INFO: Object 0xffff88033b2d6100 @offset=8448
[310365.339921] INFO: Object 0xffff88033b2d6200 @offset=8704
[310365.340324] INFO: Object 0xffff88033b2d6300 @offset=8960
[310365.340689] INFO: Object 0xffff88033b2d6400 @offset=9216
[310365.341078] INFO: Object 0xffff88033b2d6500 @offset=9472
[310365.341433] INFO: Object 0xffff88033b2d6600 @offset=9728
[310365.341798] INFO: Object 0xffff88033b2d6700 @offset=9984
[310365.342200] INFO: Object 0xffff88033b2d6800 @offset=10240
[310365.342557] INFO: Object 0xffff88033b2d6900 @offset=10496
[310365.342926] INFO: Object 0xffff88033b2d6a00 @offset=10752
[310365.343308] INFO: Object 0xffff88033b2d6b00 @offset=11008
[310365.343625] INFO: Object 0xffff88033b2d6c00 @offset=11264
[310365.343939] INFO: Object 0xffff88033b2d6d00 @offset=11520
[310365.344185] INFO: Object 0xffff88033b2d6e00 @offset=11776
[310365.344411] INFO: Object 0xffff88033b2d6f00 @offset=12032
[310365.344622] INFO: Object 0xffff88033b2d7000 @offset=12288
[310365.344863] INFO: Object 0xffff88033b2d7100 @offset=12544
[310365.345041] INFO: Object 0xffff88033b2d7200 @offset=12800
[310365.345218] INFO: Object 0xffff88033b2d7300 @offset=13056
[310365.345394] INFO: Object 0xffff88033b2d7400 @offset=13312
[310365.345571] INFO: Object 0xffff88033b2d7500 @offset=13568
[310365.345758] INFO: Object 0xffff88033b2d7600 @offset=13824
[310365.346019] INFO: Object 0xffff88033b2d7700 @offset=14080
[310365.346196] INFO: Object 0xffff88033b2d7800 @offset=14336
[310365.346371] INFO: Object 0xffff88033b2d7900 @offset=14592
[310365.346548] INFO: Object 0xffff88033b2d7a00 @offset=14848
[310365.346736] INFO: Object 0xffff88033b2d7b00 @offset=15104
[310365.346986] INFO: Object 0xffff88033b2d7c00 @offset=15360
[310365.347162] INFO: Object 0xffff88033b2d7d00 @offset=15616
[310365.347338] INFO: Object 0xffff88033b2d7f00 @offset=16128
[310365.347537] general protection fault: 0000 [#1] SMP
[310365.347729] Modules linked in: loop binfmt_misc macvlan xt_statistic xt_recent ipt_REJECT nf_reject_ipv4 ip_set fuse xt_comment xt_mark veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c sb_edac edac_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ipmi_ssif iTCO_wdt iTCO_vendor_support dcdbas mxm_wmi sg ipmi_si pcspkr ipmi_devintf ipmi_msghandler wmi mei_me mei lpc_ich shpchp acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache
[310365.349797] jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci crct10dif_pclmul libahci crct10dif_common crc32c_intel ixgbe libata tg3 megaraid_sas i2c_core mdio dca ptp pps_core dm_mirror dm_region_hash dm_log dm_mod
[310365.351230] CPU: 15 PID: 169758 Comm: dockerd Tainted: G B W ------------ 3.10.0-693.5.2.el7.x86_64 #1
[310365.351796] Hardware name: Dell Inc. PowerEdge R430/0CN7X8, BIOS 2.4.2 01/09/2017
[310365.352346] task: ffff8801f3260000 ti: ffff88091ced8000 task.ti: ffff88091ced8000
[310365.352916] RIP: 0010:[<ffffffff811e1829>] [<ffffffff811e1829>] kmem_cache_close+0xc9/0x2e0
[310365.353514] RSP: 0018:ffff88091cedbaa8 EFLAGS: 00010202
[310365.354150] RAX: ffff8817d3b76101 RBX: ffff88033b2d8000 RCX: 00000000000a4ad3
[310365.354775] RDX: 00000000000a4ad2 RSI: 0000000000000246 RDI: ffff88017fc07e00
[310365.355412] RBP: ffff88091cedbaf8 R08: 0000000000019b60 R09: ffffffff811e1945
[310365.356064] R10: ffff88203dfd9b60 R11: ffffea005f4edd80 R12: ffff881dbfacf600
[310365.356727] R13: ffffea000cecb500 R14: ffff88033b2d4000 R15: dead0000000000e0
[310365.357398] FS: 00007f5f60ff9700(0000) GS:ffff88203dfc0000(0000) knlGS:0000000000000000
[310365.358094] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[310365.358800] CR2: 00007f6df9497efc CR3: 000000053bb91000 CR4: 00000000003407e0
[310365.359523] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[310365.360256] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[310365.361025] Stack:
[310365.361763] ffff881dbfacf600 00000000811f5eaf ffff880d03e8dec0 ffff8817d3b76268
[310365.362547] ffff880d03e8ded0 ffff881dbfacf600 ffff881dbfacf600 ffff88017fc07700
[310365.363336] 0000000000000001 ffff8817a4a60600 ffff88091cedbb18 ffffffff811e1a54
[310365.364170] Call Trace:
[310365.364975] [<ffffffff811e1a54>] __kmem_cache_shutdown+0x14/0x80
[310365.365835] [<ffffffff811a67e4>] kmem_cache_destroy+0x44/0xf0
[310365.366669] [<ffffffff811f5f89>] kmem_cache_destroy_memcg_children+0x89/0xb0
[310365.367512] [<ffffffff811a67b9>] kmem_cache_destroy+0x19/0xf0
[310365.368397] [<ffffffff8123b0fe>] bioset_free+0xce/0x110
[310365.369276] [<ffffffffc000ae7e>] dm_io_client_destroy+0x1e/0x30 [dm_mod]
[310365.370155] [<ffffffffc000bff5>] dm_kcopyd_client_destroy+0xc5/0x110 [dm_mod]
[310365.371076] [<ffffffff811dee33>] ? kfree+0x103/0x140
[310365.371973] [<ffffffffc04a3b2f>] __pool_dec+0x7f/0x100 [dm_thin_pool]
[310365.372919] [<ffffffffc04a3bde>] pool_dtr+0x2e/0x70 [dm_thin_pool]
[310365.373845] [<ffffffffc00056c0>] dm_table_destroy+0x70/0x120 [dm_mod]
[310365.374781] [<ffffffffc00016b4>] __dm_destroy+0x144/0x310 [dm_mod]
[310365.375727] [<ffffffffc0003b43>] dm_destroy+0x13/0x20 [dm_mod]
[310365.376680] [<ffffffffc00099be>] dev_remove+0x11e/0x180 [dm_mod]
[310365.377642] [<ffffffffc00098a0>] ? dev_suspend+0x250/0x250 [dm_mod]
[310365.378613] [<ffffffffc000a015>] ctl_ioctl+0x1e5/0x500 [dm_mod]
[310365.379593] [<ffffffffc000a343>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[310365.380574] [<ffffffff812151bd>] do_vfs_ioctl+0x33d/0x540
[310365.381591] [<ffffffff816b0091>] ? __do_page_fault+0x171/0x450
[310365.382548] [<ffffffff81215461>] SyS_ioctl+0xa1/0xc0
[310365.383510] [<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
[310365.384418] Code: f7 75 44 e9 4a 01 00 00 66 2e 0f 1f 84 00 00 00 00 00 e8 8b c2 15 00 48 8b 45 c0 4c 89 ee 4c 89 e7 48 83 68 08 01 e8 67 b1 ff ff <49> 8b 47 20 49 8d 7f 20 48 83 e8 20 48 3b 7d d0 0f 84 11 01 00
[310365.386312] RIP [<ffffffff811e1829>] kmem_cache_close+0xc9/0x2e0
[310365.387204] RSP <ffff88091cedbaa8>
Steps To Reproducewe have not found the way to reproduce, but have happened several times with the same crash messages
Additional Informationwe use K8S-1.9.8 and docker-17.12.1
Tagscentos 7, crash, docker, kernel
abrt_hash
URL

Activities

htzk

htzk

2018-08-17 07:21

reporter  

vmcore-dmesg0817.txt (1,045,874 bytes)

Issue History

Date Modified Username Field Change
2018-08-17 07:21 htzk New Issue
2018-08-17 07:21 htzk File Added: vmcore-dmesg0817.txt
2018-08-17 07:21 htzk Tag Attached: docker
2018-08-17 07:21 htzk Tag Attached: kernel
2018-08-17 07:28 htzk Tag Attached: crash
2018-08-17 07:28 htzk Tag Attached: centos 7