View Issue Details

IDProjectCategoryView StatusLast Update
0016320CentOS-7kernelpublic2019-10-21 00:37
Reporterskkwish 
PriorityurgentSeveritymajorReproducibilitysometimes
Status newResolutionopen 
Product Version7.3.1611 
Target VersionFixed in Version 
Summary0016320: scsi_remove_device and scan cause to hang
Description  My srv is centos7.3(kernel version3.10.0-514.el7.x86_64)´╝îwhile del scsi device and scan maybe cause to hang.

Aug 5 16:15:30 openstackcompute10 kernel: INFO: task kworker/u24:6:358 blocked for more than 120 seconds.
Aug 5 16:15:30 openstackcompute10 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 5 16:15:30 openstackcompute10 kernel: kworker/u24:6 D ffff880181c11f60 0 358 2 0x00000000
Aug 5 16:15:30 openstackcompute10 kernel: Workqueue: scsi_wq_1 fc_scsi_scan_rport [scsi_transport_fc]
Aug 5 16:15:30 openstackcompute10 kernel: ffff8804683bfd30 0000000000000046 ffff88046aa59f60 ffff8804683bffd8
Aug 5 16:15:30 openstackcompute10 kernel: ffff8804683bffd8 ffff8804683bffd8 ffff88046aa59f60 ffff880468661060
Aug 5 16:15:30 openstackcompute10 kernel: ffff880468661064 ffff88046aa59f60 00000000ffffffff ffff880468661068
Aug 5 16:15:30 openstackcompute10 kernel: Call Trace:
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8168c769>] schedule_preempt_disabled+0x29/0x70
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8168a3c5>] __mutex_lock_slowpath+0xc5/0x1c0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8168982f>] mutex_lock+0x1f/0x2f
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8145dfc3>] scsi_scan_target+0x93/0x130
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffffa0052feb>] fc_scsi_scan_rport+0xbb/0xc0 [scsi_transport_fc]
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a7f3b>] process_one_work+0x17b/0x470
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a8d76>] worker_thread+0x126/0x410
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a8c50>] ? rescuer_thread+0x460/0x460
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810b052f>] kthread+0xcf/0xe0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff81696518>] ret_from_fork+0x58/0x90
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Aug 5 16:15:30 openstackcompute10 kernel: INFO: task kworker/u24:9:361 blocked for more than 120 seconds.
Aug 5 16:15:30 openstackcompute10 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 5 16:15:30 openstackcompute10 kernel: kworker/u24:9 D ffff8801822ace70 0 361 2 0x00000000
Aug 5 16:15:30 openstackcompute10 kernel: Workqueue: sysfsd sysfs_schedule_callback_work
Aug 5 16:15:30 openstackcompute10 kernel: ffff8804683dba90 0000000000000046 ffff8804683d0000 ffff8804683dbfd8
Aug 5 16:15:30 openstackcompute10 kernel: ffff8804683dbfd8 ffff8804683dbfd8 ffff8804683d0000 ffff8804683dbbd8
Aug 5 16:15:30 openstackcompute10 kernel: ffff8804683dbbe0 7fffffffffffffff ffff8804683d0000 ffff8804695fbc00
Aug 5 16:15:30 openstackcompute10 kernel: Call Trace:
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8168b679>] schedule+0x29/0x70
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff816890c9>] schedule_timeout+0x239/0x2d0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810c1989>] ? ttwu_do_wakeup+0x19/0xd0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810c1b1d>] ? ttwu_do_activate.constprop.90+0x5d/0x70
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810c4cf8>] ? try_to_wake_up+0x1c8/0x330
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8168ba56>] wait_for_completion+0x116/0x170
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810c4ec0>] ? wake_up_state+0x20/0x20
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a915c>] flush_work+0xfc/0x1c0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a50d0>] ? move_linked_works+0x90/0x90
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a92a9>] __cancel_work_timer+0x89/0x120
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a9373>] cancel_delayed_work_sync+0x13/0x20
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff812ff7c0>] disk_block_events+0x80/0x90
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff812ff8ba>] del_gendisk+0x3a/0x2a0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff81428150>] ? device_del+0x1c0/0x200
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffffa0160ff6>] sd_remove+0x56/0xb0 [sd_mod]
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8142c7bf>] __device_release_driver+0x7f/0xf0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8142c853>] device_release_driver+0x23/0x30
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8142bfc8>] bus_remove_device+0x108/0x180
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff814280c5>] device_del+0x135/0x200
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff814604e3>] __scsi_remove_device+0xc3/0xd0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff81460516>] scsi_remove_device+0x26/0x40
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff81460545>] sdev_store_delete_callback+0x15/0x20
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff8127b214>] sysfs_schedule_callback_work+0x14/0x60
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a7f3b>] process_one_work+0x17b/0x470
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a8d76>] worker_thread+0x126/0x410
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810a8c50>] ? rescuer_thread+0x460/0x460
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810b052f>] kthread+0xcf/0xe0
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff81696518>] ret_from_fork+0x58/0x90
Aug 5 16:15:30 openstackcompute10 kernel: [<ffffffff810b0460>] ? kthread_create_on_node+0x140/0x140

TagsNo tags attached.
abrt_hash
URL

Activities

TrevorH

TrevorH

2019-08-07 12:53

manager   ~0034931

Only the current version is supported. Run `yum update` to get to 7.6, reboot and try again. RHEL 7.7 was released yesterday and CentOS 7.7 will follow.
skkwish

skkwish

2019-08-07 13:04

reporter   ~0034932

But my os can't be updated.
tigalch

tigalch

2019-08-07 13:11

manager   ~0034933

None the less, what TrevorH said.
TrevorH

TrevorH

2019-08-07 13:29

manager   ~0034934

If it's CentOS then it *can* be updated.

There is no support for older versions. None. You either stay up to date or you live with the bugs that are inevitably in older software. Since you are running the original 7.3 kernel with no updates, you get to live with the several thousand bugs that have been fixed since it was released in November 2016. That's now coming on for 3 years old and is missing all the fixes for all security vulnerabilities discovered since then.
ronniela

ronniela

2019-10-17 07:53

reporter   ~0035521

This also happens to me on centos 7.6 with updated kernel

[root@stratonode0 volume_connector]# uname -r
3.10.0-957.27.2.el7.x86_64


root 1498 0.0 0.0 4360 352 ? D 05:59 0:00 tee -a /sys/block/sdn/device/delete
root 1547 0.0 0.0 4360 356 ? D 06:07 0:00 tee -a /sys/block/sdm/device/delete
root 1600 0.0 0.0 4360 356 ? D 06:41 0:00 tee -a /sys/class/scsi_host/host12/scan


[root@stratonode0 volume_connector]# cat /proc/1600/stack
[<ffffffff8f2dfc43>] scsi_scan_target+0x93/0x130
[<ffffffffc02eb32b>] iscsi_user_scan_session.part.13+0xdb/0x110 [scsi_transport_iscsi]
[<ffffffffc02eb381>] iscsi_user_scan_session+0x21/0x30 [scsi_transport_iscsi]
[<ffffffff8f2a52a5>] device_for_each_child+0x55/0x90
[<ffffffffc02e8fb3>] iscsi_user_scan+0x43/0x60 [scsi_transport_iscsi]
[<ffffffff8f2e1c58>] store_scan+0xa8/0x100
[<ffffffff8f2a479b>] dev_attr_store+0x1b/0x30
[<ffffffff8f0ccde2>] sysfs_kf_write+0x42/0x50
[<ffffffff8f0cc3bb>] kernfs_fop_write+0xeb/0x160
[<ffffffff8f042700>] vfs_write+0xc0/0x1f0
[<ffffffff8f04351f>] SyS_write+0x7f/0xf0
[<ffffffff8f576ddb>] system_call_fastpath+0x22/0x27
[<ffffffffffffffff>] 0xffffffffffffffff
[root@stratonode0 volume_connector]# cat /proc/1498/stack
[<ffffffff8eebb71d>] flush_work+0xfd/0x190
[<ffffffff8eebb839>] __cancel_work_timer+0x89/0x120
[<ffffffff8eebb903>] cancel_delayed_work_sync+0x13/0x20
[<ffffffff8f15bb50>] disk_block_events+0x80/0x90
[<ffffffff8f15bc4a>] del_gendisk+0x3a/0x2a0
[<ffffffffc038d396>] sd_remove+0x56/0xb0 [sd_mod]
[<ffffffff8f2aa222>] __device_release_driver+0x82/0xf0
[<ffffffff8f2aa2b3>] device_release_driver+0x23/0x30
[<ffffffff8f2a997b>] bus_remove_device+0x10b/0x180
[<ffffffff8f2a59ea>] device_del+0x13a/0x210
[<ffffffff8f2e2173>] __scsi_remove_device+0xd3/0xe0
[<ffffffff8f2e21a6>] scsi_remove_device+0x26/0x40
[<ffffffff8f2e21e7>] sdev_store_delete+0x27/0x30
[<ffffffff8f2a479b>] dev_attr_store+0x1b/0x30
[<ffffffff8f0ccde2>] sysfs_kf_write+0x42/0x50
[<ffffffff8f0cc3bb>] kernfs_fop_write+0xeb/0x160
[<ffffffff8f042700>] vfs_write+0xc0/0x1f0
[<ffffffff8f04351f>] SyS_write+0x7f/0xf0
[<ffffffff8f576ddb>] system_call_fastpath+0x22/0x27
[<ffffffffffffffff>] 0xffffffffffffffff
[root@stratonode0 volume_connector]# cat /proc/1547/stack
[<ffffffff8eebb71d>] flush_work+0xfd/0x190
[<ffffffff8eebb839>] __cancel_work_timer+0x89/0x120
[<ffffffff8eebb903>] cancel_delayed_work_sync+0x13/0x20
[<ffffffff8f15bb50>] disk_block_events+0x80/0x90
[<ffffffff8f15bc4a>] del_gendisk+0x3a/0x2a0
[<ffffffffc038d396>] sd_remove+0x56/0xb0 [sd_mod]
[<ffffffff8f2aa222>] __device_release_driver+0x82/0xf0
[<ffffffff8f2aa2b3>] device_release_driver+0x23/0x30
[<ffffffff8f2a997b>] bus_remove_device+0x10b/0x180
[<ffffffff8f2a59ea>] device_del+0x13a/0x210
[<ffffffff8f2e2173>] __scsi_remove_device+0xd3/0xe0
[<ffffffff8f2e21a6>] scsi_remove_device+0x26/0x40
[<ffffffff8f2e21e7>] sdev_store_delete+0x27/0x30
[<ffffffff8f2a479b>] dev_attr_store+0x1b/0x30
[<ffffffff8f0ccde2>] sysfs_kf_write+0x42/0x50
[<ffffffff8f0cc3bb>] kernfs_fop_write+0xeb/0x160
[<ffffffff8f042700>] vfs_write+0xc0/0x1f0
[<ffffffff8f04351f>] SyS_write+0x7f/0xf0
[<ffffffff8f576ddb>] system_call_fastpath+0x22/0x27
[<ffffffffffffffff>] 0xffffffffffffffff

Issue History

Date Modified Username Field Change
2019-08-07 12:21 skkwish New Issue
2019-08-07 12:53 TrevorH Note Added: 0034931
2019-08-07 13:04 skkwish Note Added: 0034932
2019-08-07 13:11 tigalch Note Added: 0034933
2019-08-07 13:29 TrevorH Note Added: 0034934
2019-10-17 07:53 ronniela Note Added: 0035521