View Issue Details

IDProjectCategoryView StatusLast Update
0005764CentOS-6kernelpublic2012-06-05 09:09
Reportertomasz_asb 
PriorityhighSeveritycrashReproducibilityrandom
Status newResolutionopen 
Platformx86_64OSCentOSOS Version6.0 (Final)
Product Version6.0 
Target VersionFixed in Version 
Summary0005764: CPU soft lockup, system crashed
DescriptionAfter several weeks of uptime, under low to moderate loads, the CentOS machine crashed. The OS would respond to ping, but nothing else worked (including SSH and physical console).
Steps To ReproduceUnknown. I will be migrating the machine to VMWare ESXi next, so reproducing will be impossible.
Additional InformationJun 3 03:43:01 rms1 kernel: imklog 4.6.2, log source = /proc/kmsg started.
Jun 3 03:43:01 rms1 rsyslogd: [origin software="rsyslogd" swVersion="4.6.2" x-pid="2094" x-info="http://www.rsyslog.com"] (re)start
Jun 5 03:15:47 rms1 kernel: BUG: soft lockup - CPU#0 stuck for 17163091968s! [java:32285]
Jun 5 03:15:47 rms1 kernel: Modules linked in: autofs4 sunrpc ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables nf_conntrack_netbios_ns ip6t_REJECT nf_conntrack_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log power_meter hwmon dcdbas serio_raw iTCO_wdt iTCO_vendor_support i7core_edac edac_core igb dca sg bnx2 ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix megaraid_sas dm_mod [last unloaded: speedstep_lib]
Jun 5 03:15:47 rms1 kernel: CPU 0:
Jun 5 03:15:47 rms1 kernel: Modules linked in: autofs4 sunrpc ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables nf_conntrack_netbios_ns ip6t_REJECT nf_conntrack_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log power_meter hwmon dcdbas serio_raw iTCO_wdt iTCO_vendor_support i7core_edac edac_core igb dca sg bnx2 ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix megaraid_sas dm_mod [last unloaded: speedstep_lib]
Jun 5 03:15:47 rms1 kernel: Pid: 32285, comm: java Not tainted 2.6.32-71.29.1.el6.x86_64 #1 PowerEdge R510
Jun 5 03:15:47 rms1 kernel: RIP: 0033:[<00007fe3690761ac>] [<00007fe3690761ac>] 0x7fe3690761ac
Jun 5 03:15:47 rms1 kernel: RSP: 002b:00007fe10f2f1120 EFLAGS: 00000286
Jun 5 03:15:47 rms1 kernel: RAX: 000000002aec3ea0 RBX: 00000000c683c9b8 RCX: 00000000c85efb60
Jun 5 03:15:47 rms1 kernel: RDX: 0000000032eed8b9 RSI: 00000000b75206fc RDI: 0000000000000035
Jun 5 03:15:47 rms1 kernel: RBP: ffffffff81013c8e R08: 0000000002aee92c R09: 000000002aee92cd
Jun 5 03:15:47 rms1 kernel: R10: 00000000c683c9b8 R11: 00000000b2e826c4 R12: 00000000c71a5d88
Jun 5 03:15:47 rms1 kernel: R13: 00007fe104011000 R14: 0000000000000000 R15: 00007fe10f2f1160
Jun 5 03:15:47 rms1 kernel: FS: 00007fe10f2f2700(0000) GS:ffff880237400000(0000) knlGS:0000000000000000
Jun 5 03:15:47 rms1 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Jun 5 03:15:47 rms1 kernel: CR2: 00007fe371efd000 CR3: 00000001676a5000 CR4: 00000000000006f0
Jun 5 03:15:47 rms1 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jun 5 03:15:47 rms1 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jun 5 03:15:47 rms1 kernel: Call Trace:
Jun 5 03:15:47 rms1 kernel: BUG: soft lockup - CPU#1 stuck for 17163091968s! [java:32301]
Jun 5 03:15:47 rms1 kernel: Modules linked in: autofs4 sunrpc ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables nf_conntrack_netbios_ns ip6t_REJECT nf_conntrack_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log power_meter hwmon dcdbas serio_raw iTCO_wdt iTCO_vendor_support i7core_edac edac_core igb dca sg bnx2 ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix megaraid_sas dm_mod [last unloaded: speedstep_lib]
Jun 5 03:15:47 rms1 kernel: CPU 1:
Jun 5 03:15:47 rms1 kernel: Modules linked in: autofs4 sunrpc ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables nf_conntrack_netbios_ns ip6t_REJECT nf_conntrack_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log power_meter hwmon dcdbas serio_raw iTCO_wdt iTCO_vendor_support i7core_edac edac_core igb dca sg bnx2 ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix megaraid_sas dm_mod [last unloaded: speedstep_lib]
Jun 5 03:15:47 rms1 kernel: Pid: 32301, comm: java Not tainted 2.6.32-71.29.1.el6.x86_64 #1 PowerEdge R510
Jun 5 03:15:47 rms1 kernel: RIP: 0033:[<00007fe3690fb27a>] [<00007fe3690fb27a>] 0x7fe3690fb27a
Jun 5 03:15:47 rms1 kernel: RSP: 002b:00007fe10e8e7020 EFLAGS: 00000286
Jun 5 03:15:47 rms1 kernel: RAX: 0000000065254c43 RBX: 00000000c5651Jun 5 09:58:12 rms1 kernel: imklog 4.6.2, log source = /proc/kmsg started.


processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
stepping : 2
cpu MHz : 2666.288
cache size : 12288 KB
physical id : 1
siblings : 12
core id : 0
cpu cores : 6
apicid : 32
initial apicid : 32
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 5332.57
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
stepping : 2
cpu MHz : 2666.288
cache size : 12288 KB
physical id : 0
siblings : 12
core id : 0
cpu cores : 6
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 5333.25
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

... etc. - for a total of 24 CPUs visible.


TagsNo tags attached.

Activities

There are no notes attached to this issue.

Issue History

Date Modified Username Field Change
2012-06-05 09:09 tomasz_asb New Issue