View Issue Details

IDProjectCategoryView StatusLast Update
0018433CentOS-7kernelpublic2022-04-15 10:25
Reporterhejiwen Assigned To 
PriorityhighSeveritymajorReproducibilityalways
Status newResolutionopen 
Platformintel 
Product Version7.9.2009 
Summary0018433: sync process enters uninterrupted state
Descriptionkernel-3.10.0-1160.62.1



Steps To Reproduce1. create many sparese file,size=500G
2. use fio to write this files
3. when disk to fault sector,the process enter uninterrupted state, if exec sync command,sync also enter uninterrupted state,call strace as follows:
TagsNo tags attached.
abrt_hash
URL

Activities

hejiwen

hejiwen

2022-04-15 02:47

reporter   ~0038904

Apr 15 02:34:53 inspur01 kernel: sd 14:0:6:0: [sdd] tag#11 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Apr 15 02:34:53 inspur01 kernel: sd 14:0:6:0: [sdd] tag#11 Sense Key : Hardware Error [current]
Apr 15 02:34:53 inspur01 kernel: sd 14:0:6:0: [sdd] tag#11 Add. Sense: Mechanical positioning error
Apr 15 02:34:53 inspur01 kernel: sd 14:0:6:0: [sdd] tag#11 CDB: Write(10) 2a 00 08 98 b4 f0 00 02 00 00
Apr 15 02:34:53 inspur01 kernel: blk_update_request: I/O error, dev sdd, sector 144225934
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716382, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716383, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716384, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716385, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716386, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716387, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716388, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716389, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716390, lost async page write
Apr 15 02:34:53 inspur01 kernel: Buffer I/O error on dev sdd1, logical block 16716391, lost async page write
Apr 15 02:37:45 inspur01 kernel: INFO: task filestore_sync:8998 blocked for more than 120 seconds.
Apr 15 02:37:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:37:45 inspur01 kernel: <NVDIMM> Read ADR status from CMOS:0x00[jedec_nvdimm_init][831]
Apr 15 02:37:45 inspur01 kernel: filestore_sync D
Apr 15 02:37:45 inspur01 kernel: ffff882034f51fa0 0 8998 1 0x00000080
Apr 15 02:37:45 inspur01 kernel: Call Trace:
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff810d546e>] ? dequeue_task_fair+0x41e/0x660
Apr 15 02:37:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:37:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:37:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:37:45 inspur01 kernel: [<ffffffff810ad74a>] ? __queue_delayed_work+0xaa/0x1a0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff810adab1>] ? try_to_grab_pending+0xb1/0x160
Apr 15 02:37:45 inspur01 kernel: [<ffffffff8134a43d>] ? list_del+0xd/0x30
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816bbbfd>] ? wait_for_completion+0x11d/0x140
Apr 15 02:37:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:37:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:37:45 inspur01 kernel: INFO: task journal_write:9216 blocked for more than 120 seconds.
Apr 15 02:37:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:37:45 inspur01 kernel: journal_write D ffff88201bf38fd0 0 9216 1 0x00000080
Apr 15 02:37:45 inspur01 kernel: Call Trace:
Apr 15 02:37:45 inspur01 kernel: [<ffffffffc08561f1>] ? xfs_inode_item_format_attr_fork.isra.7+0xb1/0x270 [xfs]
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff811449ef>] ? delayacct_end+0x8f/0xb0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:37:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:37:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:37:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:37:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:37:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:39:45 inspur01 kernel: INFO: task filestore_sync:8998 blocked for more than 120 seconds.
Apr 15 02:39:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:39:45 inspur01 kernel: filestore_sync D ffff882034f51fa0 0 8998 1 0x00000080
Apr 15 02:39:45 inspur01 kernel: Call Trace:
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff810d546e>] ? dequeue_task_fair+0x41e/0x660
Apr 15 02:39:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:39:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:39:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:39:45 inspur01 kernel: [<ffffffff810ad74a>] ? __queue_delayed_work+0xaa/0x1a0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff810adab1>] ? try_to_grab_pending+0xb1/0x160
Apr 15 02:39:45 inspur01 kernel: [<ffffffff8134a43d>] ? list_del+0xd/0x30
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816bbbfd>] ? wait_for_completion+0x11d/0x140
Apr 15 02:39:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:39:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:39:45 inspur01 kernel: INFO: task journal_write:9216 blocked for more than 120 seconds.
Apr 15 02:39:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:39:45 inspur01 kernel: journal_write D ffff88201bf38fd0 0 9216 1 0x00000080
Apr 15 02:39:45 inspur01 kernel: Call Trace:
Apr 15 02:39:45 inspur01 kernel: [<ffffffffc08561f1>] ? xfs_inode_item_format_attr_fork.isra.7+0xb1/0x270 [xfs]
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff811449ef>] ? delayacct_end+0x8f/0xb0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:39:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:39:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:39:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:39:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:39:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:41:45 inspur01 kernel: INFO: task filestore_sync:8998 blocked for more than 120 seconds.
Apr 15 02:41:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:41:45 inspur01 kernel: filestore_sync D ffff882034f51fa0 0 8998 1 0x00000080
Apr 15 02:41:45 inspur01 kernel: Call Trace:
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff810d546e>] ? dequeue_task_fair+0x41e/0x660
Apr 15 02:41:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:41:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:41:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:41:45 inspur01 kernel: [<ffffffff810ad74a>] ? __queue_delayed_work+0xaa/0x1a0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff810adab1>] ? try_to_grab_pending+0xb1/0x160
Apr 15 02:41:45 inspur01 kernel: [<ffffffff8134a43d>] ? list_del+0xd/0x30
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816bbbfd>] ? wait_for_completion+0x11d/0x140
Apr 15 02:41:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:41:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:41:45 inspur01 kernel: INFO: task journal_write:9216 blocked for more than 120 seconds.
Apr 15 02:41:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:41:45 inspur01 kernel: journal_write D ffff88201bf38fd0 0 9216 1 0x00000080
Apr 15 02:41:45 inspur01 kernel: Call Trace:
Apr 15 02:41:45 inspur01 kernel: [<ffffffffc08561f1>] ? xfs_inode_item_format_attr_fork.isra.7+0xb1/0x270 [xfs]
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff811449ef>] ? delayacct_end+0x8f/0xb0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:41:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:41:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:41:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:41:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:41:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:43:45 inspur01 kernel: INFO: task filestore_sync:8998 blocked for more than 120 seconds.
Apr 15 02:43:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:43:45 inspur01 kernel: filestore_sync D ffff882034f51fa0 0 8998 1 0x00000080
Apr 15 02:43:45 inspur01 kernel: Call Trace:
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff810d546e>] ? dequeue_task_fair+0x41e/0x660
Apr 15 02:43:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:43:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:43:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:43:45 inspur01 kernel: [<ffffffff810ad74a>] ? __queue_delayed_work+0xaa/0x1a0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff810adab1>] ? try_to_grab_pending+0xb1/0x160
Apr 15 02:43:45 inspur01 kernel: [<ffffffff8134a43d>] ? list_del+0xd/0x30
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816bbbfd>] ? wait_for_completion+0x11d/0x140
Apr 15 02:43:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:43:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:43:45 inspur01 kernel: INFO: task journal_write:9216 blocked for more than 120 seconds.
Apr 15 02:43:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:43:45 inspur01 kernel: journal_write D ffff88201bf38fd0 0 9216 1 0x00000080
Apr 15 02:43:45 inspur01 kernel: Call Trace:
Apr 15 02:43:45 inspur01 kernel: [<ffffffffc08561f1>] ? xfs_inode_item_format_attr_fork.isra.7+0xb1/0x270 [xfs]
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff811449ef>] ? delayacct_end+0x8f/0xb0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:43:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:43:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:43:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:43:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:43:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:45:45 inspur01 kernel: INFO: task filestore_sync:8998 blocked for more than 120 seconds.
Apr 15 02:45:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:45:45 inspur01 kernel: filestore_sync D ffff882034f51fa0 0 8998 1 0x00000084
Apr 15 02:45:45 inspur01 kernel: Call Trace:
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff810d546e>] ? dequeue_task_fair+0x41e/0x660
Apr 15 02:45:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:45:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:45:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:45:45 inspur01 kernel: [<ffffffff810ad74a>] ? __queue_delayed_work+0xaa/0x1a0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff810adab1>] ? try_to_grab_pending+0xb1/0x160
Apr 15 02:45:45 inspur01 kernel: [<ffffffff8134a43d>] ? list_del+0xd/0x30
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816bbbfd>] ? wait_for_completion+0x11d/0x140
Apr 15 02:45:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:45:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
Apr 15 02:45:45 inspur01 kernel: INFO: task journal_write:9216 blocked for more than 120 seconds.
Apr 15 02:45:45 inspur01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 15 02:45:45 inspur01 kernel: journal_write D ffff88201bf38fd0 0 9216 1 0x00000084
Apr 15 02:45:45 inspur01 kernel: Call Trace:
Apr 15 02:45:45 inspur01 kernel: [<ffffffffc08561f1>] ? xfs_inode_item_format_attr_fork.isra.7+0xb1/0x270 [xfs]
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816bb829>] schedule+0x29/0x70
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b91e9>] schedule_timeout+0x239/0x2c0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff811449ef>] ? delayacct_end+0x8f/0xb0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff810f03d2>] ? ktime_get_ts64+0x52/0xf0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b9800>] ? bit_wait+0x50/0x50
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816bad8d>] io_schedule_timeout+0xad/0x130
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816bae28>] io_schedule+0x18/0x20
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b9811>] bit_wait_io+0x11/0x50
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816b9337>] __wait_on_bit+0x67/0x90
Apr 15 02:45:45 inspur01 kernel: [<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff810b8520>] ? wake_bit_function+0x40/0x40
Apr 15 02:45:45 inspur01 kernel: [<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
Apr 15 02:45:45 inspur01 kernel: [<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
Apr 15 02:45:45 inspur01 kernel: [<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff8123e4b6>] sync_filesystem+0x66/0xb0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff8123e664>] SyS_syncfs+0x54/0xa0
Apr 15 02:45:45 inspur01 kernel: [<ffffffff816c86db>] system_call_fastpath+0x22/0x27
hejiwen

hejiwen

2022-04-15 02:47

reporter   ~0038905

977061 root 20 0 108052 340 264 D 0.0 0.0 0:00.00 sync
[root@localhost ~]# cat /proc/977061/stack
[<ffffffff81189741>] wait_on_page_bit+0x81/0xa0
[<ffffffff81189871>] __filemap_fdatawait_range+0x111/0x190
[<ffffffff8118c6a7>] filemap_fdatawait_keep_errors+0x27/0x30
[<ffffffff81238bbd>] sync_inodes_sb+0x16d/0x1f0
[<ffffffff8123e269>] sync_inodes_one_sb+0x19/0x20
[<ffffffff8120f833>] iterate_supers+0xc3/0x120
[<ffffffff8123e544>] sys_sync+0x44/0xb0
[<ffffffff816c86db>] system_call_fastpath+0x22/0x27
[<ffffffffffffffff>] 0xffffffffffffffff
hejiwen

hejiwen

2022-04-15 02:49

reporter   ~0038906

must reboot to solve this problem
ManuelWolfshant

ManuelWolfshant

2022-04-15 06:42

manager   ~0038907

You have faulty hardware ( see the very first 10 lines of your logs ). Nothing that you have included later shows a loop, the system just can't recover past the defective disk
hejiwen

hejiwen

2022-04-15 09:58

reporter   ~0038908

Thanks for your reply。
I update kernel to 4.18, this problem havn't happen,It can tolerate this error, But I can't update kernel in production environment。
I want to know how to find the patch fixed this problem,can you give me some advice?
hejiwen

hejiwen

2022-04-15 10:25

reporter   ~0038910

4.18 dmesg info
111.png (53,528 bytes)   
111.png (53,528 bytes)   

Issue History

Date Modified Username Field Change
2022-04-15 02:43 hejiwen New Issue
2022-04-15 02:47 hejiwen Note Added: 0038904
2022-04-15 02:47 hejiwen Note Added: 0038905
2022-04-15 02:49 hejiwen Note Added: 0038906
2022-04-15 06:42 ManuelWolfshant Note Added: 0038907
2022-04-15 09:58 hejiwen Note Added: 0038908
2022-04-15 10:25 hejiwen Note Added: 0038910
2022-04-15 10:25 hejiwen File Added: 111.png