View Issue Details

IDProjectCategoryView StatusLast Update
0014427CentOS-7kernelpublic2019-01-14 15:44
Reportercarlosadean 
PriorityurgentSeveritymajorReproducibilityrandom
Status newResolutionopen 
Product Version7.3.1611 
Target VersionFixed in Version 
Summary0014427: kernel panic at __kmalloc_track_caller+0x95/0x230
DescriptionIt has been happening from time to time in one VM (KVM) normally when the system has low memory avaliable, the machine crashes and reboot itself. Also, there was an app trying to restart an usb device at the same time this app was using ~70% of the memory when it happened. This output below was created by abrt-cli.

I can see a similar error at https://access.redhat.com/solutions/1404013 whitout solution yet.

from backtrace:

general protection fault: 0000 [#1] SMP
Modules linked in: fuse unix_diag tcp_diag inet_diag binfmt_misc nfsv3 nfs fscache ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd pcspkr virtio_balloon parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi virtio_blk virtio_console virtio_net floppy cirrus drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect serio_raw sysimgblt ata_piix crc32c_intel fb_sys_fops ttm drm libata virtio_pci virtio_ring i2c_core virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 5881 Comm: python Not tainted 3.10.0-514.10.2.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff88031d2d3ec0 ti: ffff880119460000 task.ti: ffff880119460000
RIP: 0010:[<ffffffff811dffe5>] [<ffffffff811dffe5>] __kmalloc_track_caller+0x95/0x230
RSP: 0018:ffff880119463b38 EFLAGS: 00010286
RAX: 0000000000000000 RBX: 0000000000000008 RCX: 00000000007d2a3d
RDX: 00000000007d2a3c RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff880119463b70 R08: 0000000000019a20 R09: ffff88017fc01e00
R10: 0000000000000008 R11: ffff88013b71a184 R12: 00000000000000d0
R13: 48ffff8800365f79 R14: 0000000000000008 R15: ffff88017fc01e00
FS: 00007fe95d316740(0000) GS:ffff88032dd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000520b2003 CR3: 000000015562e000 CR4: 00000000001406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
 ffff88017fc01e00 ffffffffa04e3b02 0000000000000008 ffff88013b71a184
 ffffea000724d800 ffff8801c9360000 ffff880119463d28 ffff880119463b90
 ffffffff8119f0b0 ffff880119463e68 0000000000000003 ffff880119463c98
Call Trace:
 [<ffffffffa04e3b02>] ? nfs_readdir_page_filler+0x2f2/0x5b0 [nfs]
 [<ffffffff8119f0b0>] kmemdup+0x20/0x50
 [<ffffffffa04e3b02>] nfs_readdir_page_filler+0x2f2/0x5b0 [nfs]
 [<ffffffffa04e3fcd>] nfs_readdir_xdr_to_array+0x20d/0x3b0 [nfs]
 [<ffffffff8118aec6>] ? __alloc_pages_nodemask+0x176/0x420
 [<ffffffffa04e4170>] ? nfs_readdir_xdr_to_array+0x3b0/0x3b0 [nfs]
 [<ffffffffa04e4192>] nfs_readdir_filler+0x22/0x90 [nfs]
 [<ffffffff8118125f>] do_read_cache_page+0x7f/0x190
 [<ffffffff81212530>] ? fillonedir+0xe0/0xe0
 [<ffffffff811813ac>] read_cache_page+0x1c/0x30
 [<ffffffffa04e43db>] nfs_readdir+0x1db/0x6b0 [nfs]
 [<ffffffffa03ecbb0>] ? nfs3_xdr_dec_getattr3res+0x80/0x80 [nfsv3]
 [<ffffffff81212530>] ? fillonedir+0xe0/0xe0
 [<ffffffff81212420>] vfs_readdir+0xb0/0xe0
 [<ffffffff81212845>] SyS_getdents+0x95/0x120
 [<ffffffff81696b09>] system_call_fastpath+0x16/0x1b
Code: d0 00 00 49 8b 50 08 4d 8b 28 49 8b 40 10 4d 85 ed 0f 84 20 01 00 00 48 85 c0 0f 84 17 01 00 00 49 63 47 20 48 8d 4a 01 4d 8b 07 <49> 8b 5c 05 00 4c 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 b8 49
RIP [<ffffffff811dffe5>] __kmalloc_track_caller+0x95/0x230
 RSP <ffff880119463b38>


from vmcore-dmesg.txt:

[2408779.877811] Out of memory: Kill process 29334 (python) score 834 or sacrifice child
[2408779.877885] Killed process 29334 (python) total-vm:9764928kB, anon-rss:6364472kB, file-rss:780kB, shmem-rss:0kB
[2410491.594856] python[2103]: segfault at 18 ip 00007fd8d054101d sp 00007ffc6b9b4730 error 4 in VimbaGigETL.cti[7fd8d04f9000+a0000]
[2704262.601116] usb 1-3: USB disconnect, device number 15
[3748370.544914] usb 1-4: new high-speed USB device number 16 using ehci-pci
[3748370.848827] usb 1-4: New USB device found, idVendor=136e, idProduct=000c
[3748370.848853] usb 1-4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[3757493.529500] usb 1-4: USB disconnect, device number 16
[3758393.420557] usb 1-5: new high-speed USB device number 17 using ehci-pci
[3758393.720956] usb 1-5: New USB device found, idVendor=136e, idProduct=000c
[3758393.720962] usb 1-5: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[3760531.985197] usb 1-5: USB disconnect, device number 17
[3760533.412606] usb 1-6: new high-speed USB device number 18 using ehci-pci
[3760533.699051] usb 1-6: config 1 interface 0 altsetting 0 bulk endpoint 0x1 has invalid maxpacket 64
[3760533.701081] usb 1-6: New USB device found, idVendor=136e, idProduct=000c
[3760533.701086] usb 1-6: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[3760533.701089] usb 1-6: Product: USB CAM
[3760533.701092] usb 1-6: Manufacturer: ANDOR
[3760533.723441] usb 1-6: usbfs: interface 0 claimed by usbfs while 'python' sets config #1
[3760533.734346] usb 1-6: usbfs: interface 0 claimed by usbfs while 'python' sets config #1
[3840490.630822] usb 1-6: USB disconnect, device number 18
[4445054.331735] usb 1-1: new high-speed USB device number 19 using ehci-pci
[4445054.632730] usb 1-1: New USB device found, idVendor=136e, idProduct=000c
[4445054.632736] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[4445133.710295] usb 1-1: USB disconnect, device number 19
[4445135.389629] usb 1-2: new high-speed USB device number 20 using ehci-pci
[4445135.674247] usb 1-2: config 1 interface 0 altsetting 0 bulk endpoint 0x1 has invalid maxpacket 64
[4445135.675748] usb 1-2: New USB device found, idVendor=136e, idProduct=000c
[4445135.675753] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[4445135.675757] usb 1-2: Product: USB CAM
[4445135.675759] usb 1-2: Manufacturer: ANDOR
[4445135.687434] usb 1-2: usbfs: interface 0 claimed by usbfs while 'python' sets config #1
[4445135.700433] usb 1-2: usbfs: interface 0 claimed by usbfs while 'python' sets config #1
[4577710.372066] general protection fault: 0000 [#1] SMP
[4577710.372142] Modules linked in: fuse unix_diag tcp_diag inet_diag binfmt_misc nfsv3 nfs fscache ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd pcspkr virtio_balloon parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi virtio_blk virtio_console virtio_net floppy cirrus drm_kms_helper crct10dif_pclmul crct10dif_common syscopyarea sysfillrect serio_raw sysimgblt ata_piix crc32c_intel fb_sys_fops ttm drm libata virtio_pci virtio_ring i2c_core virtio dm_mirror dm_region_hash dm_log dm_mod
[4577710.372840] CPU: 3 PID: 5881 Comm: python Not tainted 3.10.0-514.10.2.el7.x86_64 #1
[4577710.372909] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[4577710.372961] task: ffff88031d2d3ec0 ti: ffff880119460000 task.ti: ffff880119460000
[4577710.373026] RIP: 0010:[<ffffffff811dffe5>] [<ffffffff811dffe5>] __kmalloc_track_caller+0x95/0x230
[4577710.373111] RSP: 0018:ffff880119463b38 EFLAGS: 00010286
Steps To ReproduceI do not know hot to reproduce this error.
TagsNo tags attached.
abrt_hash
URL

Activities

toracat

toracat

2018-01-29 18:59

manager   ~0031092

What is the kernel version? You indicated "7.3.1611". According to the RH article you referenced, the issue is supposed to have been fixed in EL7.4.
tru

tru

2018-01-29 19:17

administrator   ~0031094

the reporter is using 3.10.0-514.10.2.el7.x86_64 .... from 7.3.1611 oudated 7.4.1708 has been released :)
carlosadean

carlosadean

2018-01-31 14:11

reporter   ~0031134

Here is the result of the 'uname -a':

Linux 3.10.0-514.10.2.el7.x86_64 #1 SMP Fri Mar 3 00:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

I read 'Solution In Progress' in that article. I'll update the system.
carlosadean

carlosadean

2018-02-01 13:04

reporter   ~0031143

I am not able to read the entire article, is there any release notes where I can confirm this information? Thanks.
carlosadean

carlosadean

2018-03-29 19:57

reporter   ~0031527

I have upgraded the system to EL7.4 - kernel 3.10.0-693.17.1.el7.x86_64

As I said in my first message it happens from time to time and during the last runner the bug happened again.

[ 161.349987] random: crng init done
[93050.273947] usb 1-1: new high-speed USB device number 2 using ehci-pci
[93050.573395] usb 1-1: New USB device found, idVendor=136e, idProduct=000c
[93050.573402] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[93460.753956] usb 1-1: USB disconnect, device number 2
[93462.431958] usb 1-2: new high-speed USB device number 3 using ehci-pci
[93462.732518] usb 1-2: config 1 interface 0 altsetting 0 bulk endpoint 0x1 has invalid maxpacket 64
[93462.740852] usb 1-2: New USB device found, idVendor=136e, idProduct=000c
[93462.740858] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[93462.740861] usb 1-2: Product: USB CAM
[93462.740864] usb 1-2: Manufacturer: ANDOR
[93462.764761] usb 1-2: usbfs: interface 0 claimed by usbfs while 'python' sets config #1
[93462.775697] usb 1-2: usbfs: interface 0 clae possible imed by usbfs while 'python' sets config #1
[114417.598228] general protection fault: 0000 [#1] SMP
[114417.598290] Modules linked in: nfsv3 nfs fscache ext4 mbcache jbd2 iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper parport_pc cryptd parport joydev i2c_piix4 pcspkr virtio_balloon nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c ata_generic pata_acpi cirrus drm_kms_helper virtio_blk syscopyarea sysfillrect sysimgblt fb_sys_fops virtio_console ttm virtio_net drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel floppy serio_raw virtio_pci virtio_ring virtio i2c_core dm_mirror dm_region_hash dm_log dm_mod
[114417.598932] CPU: 2 PID: 21363 Comm: python Not tainted 3.10.0-693.17.1.el7.x86_64 #1
[114417.599004] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[114417.599058] task: ffff8801dcbf4f10 ti: ffff8801ba200000 task.ti: ffff8801ba200000
[114417.599127] RIP: 0010:[<ffffffff811e3fe4>] [<ffffffff811e3fe4>] __kmalloc_track_caller+0x94/0x230
[114417.599223] RSP: 0018:ffff8801ba203b10 EFLAGS: 00010286
[114417.599273] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 000000000002e2fc
[114417.599339] RDX: 000000000002e2fb RSI: 0000000000000000 RDI: 0000000000000000
[114417.599404] RBP: ffff8801ba203b48 R08: 000000000001b8a0 R09: ffff88017fc01e00
[114417.599469] R10: ffff88022af52010 R11: ffff8801ba203bf8 R12: 00000000000000d0
[114417.599535] R13: 3100302e313a322d R14: 0000000000000001 R15: ffff88017fc01e00
[114417.599601] FS: 00007f336cefe700(0000) GS:ffff88023fd00000(0000) knlGS:0000000000000000
[114417.599689] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[114417.599743] CR2: 00007f33a324c000 CR3: 00000000b455c000 CR4: 00000000001606e0
[114417.599813] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[114417.599879] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[114417.599944] Call Trace:
[114417.599981] [<ffffffffc050e352>] ? nfs_readdir_page_filler+0x2f2/0x600 [nfs]
[114417.600051] [<ffffffff811a2bb0>] kmemdup+0x20/0x50
[114417.600103] [<ffffffffc050e352>] nfs_readdir_page_filler+0x2f2/0x600 [nfs]
[114417.600174] [<ffffffffc05142cb>] ? nfs_refresh_inode_locked+0x5b/0x2f0 [nfs]
[114417.600247] [<ffffffffc050ebbd>] nfs_readdir_xdr_to_array+0x20d/0x3b0 [nfs]
[114417.600314] [<ffffffff8118e9f6>] ? __alloc_pages_nodemask+0x176/0x420
[114417.600380] [<ffffffffc050ed60>] ? nfs_readdir_xdr_to_array+0x3b0/0x3b0 [nfs]
[114417.600452] [<ffffffffc050ed82>] nfs_readdir_filler+0x22/0x90 [nfs]
[114417.600514] [<ffffffff8118532f>] do_read_cache_page+0x7f/0x190
[114417.600572] [<ffffffff81217790>] ? fillonedir+0xe0/0xe0
[114417.600624] [<ffffffff8118547c>] read_cache_page+0x1c/0x30
[114417.600697] [<ffffffffc050ef9b>] nfs_readdir+0x1ab/0x6b0 [nfs]
[114417.600758] [<ffffffffc04fcbe0>] ? nfs3_xdr_dec_getattr3res+0x80/0x80 [nfsv3]
[114417.600826] [<ffffffff81217790>] ? fillonedir+0xe0/0xe0
[114417.600878] [<ffffffff81217680>] vfs_readdir+0xb0/0xe0
[114417.600930] [<ffffffff81217aa5>] SyS_getdents+0x95/0x120
[114417.600983] [<ffffffff816b89fd>] system_call_fastpath+0x16/0x1b
[114417.601039] Code: d1 e2 7e 49 8b 50 08 4d 8b 28 49 8b 40 10 4d 85 ed 0f 84 21 01 00 00 48 85 c0 0f 84 18 01 00 00 49 63 47 20 48 8d 4a 01 4d 8b 07 <49> 8b 5c 05 00 4c 89 e8 65 49 0f c7 08 0f 94 c0 84 c0 74 b9 49
[114417.601374] RIP [<ffffffff811e3fe4>] __kmalloc_track_caller+0x94/0x230
[114417.601435] RSP <ffff8801ba203b10>
carlosadean

carlosadean

2019-01-14 15:44

reporter   ~0033583

The bug is still there, but an adopted workaround was to install the app and to plug the usb camera into another machine.

Issue History

Date Modified Username Field Change
2018-01-29 18:46 carlosadean New Issue
2018-01-29 18:59 toracat Note Added: 0031092
2018-01-29 19:17 tru Note Added: 0031094
2018-01-31 14:11 carlosadean Note Added: 0031134
2018-02-01 13:04 carlosadean Note Added: 0031143
2018-03-29 19:57 carlosadean Note Added: 0031527
2019-01-14 15:44 carlosadean Note Added: 0033583