View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0017742 | CentOS-8 | kernel | public | 2020-09-20 18:59 | 2020-09-22 22:58 |
Reporter | GordonW | Assigned To | |||
Priority | normal | Severity | major | Reproducibility | always |
Status | new | Resolution | open | ||
Product Version | 8.1.1911 | ||||
Summary | 0017742: NFS Reports Stale File Handle after kernel-4.18.0-193.6.3.el8_2.x86_64 | ||||
Description | NFS Reports Stale File Handle after kernel-4.18.0-193.6.3.el8_2.x86_64 but no seen when reboot to kernel-4.18.0-147.8.1.el8_1.x86_64. rsync reports: [root@CentOS-8 ~]# ./Backup_rsync_Stale_Handle_Test building file list ... done ./ Fedora_26-disk001.vmdk rsync: write failed on "/mnt/HomeNAS/Linux_Backups/CentOS8_Test/Fedora_26-disk001.vmdk": Stale file handle (116) rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3] [root@CentOS-8 ~]# cp reports: [root@CentOS-8 ~]# cp /root/rsync_test/Fedora_26-disk001.vmdk /mnt/HomeNAS/Linux_Backups/CentOS8_Test/Fedora_26-disk001.vmdk cp: failed to close '/mnt/HomeNAS/Linux_Backups/CentOS8_Test/Fedora_26-disk001.vmdk': Stale file handle [root@CentOS-8 ~]# | ||||
Steps To Reproduce | Created test file: Actually made a copy of a vm_disk: [root@CentOS-8 ~]# ls -lsah ./rsync_test/ total 12G 0 drwxr-xr-x. 2 root root 36 Sep 20 11:31 . 4.0K dr-xr-x---. 16 root root 4.0K Sep 20 12:11 .. 12G -rw-------. 1 root root 12G Sep 20 11:32 Fedora_26-disk001.vmdk [root@CentOS-8 ~]# use cp command or this test rsync script [root@CentOS-8 ~]# cat ./Backup_rsync_Stale_Handle_Test #!/bin/bash # Dest="/mnt/HomeNAS/Linux_Backups/CentOS8_Test/" # /usr/bin/rsync -aurlv --no-links --stats --delete-before /root/rsync_test/ ${Dest} [root@CentOS-8 ~]# | ||||
Additional Information | NFS Version 3 automounted off Synology NAS system: [root@CentOS-8 ~]# mount |grep NAS /etc/auto.synology on /mnt/HomeNAS type autofs (rw,relatime,fd=12,pgrp=1657,timeout=600,minproto=5,maxproto=5,indirect,pipe_ino=35241) 192.168.0.9:/volume1/Linux Backups on /mnt/HomeNAS/Linux_Backups type nfs (rw,relatime,vers=3,rsize=8192,wsize=8192,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.9,mountvers=3,mountport=892,mountproto=udp,local_lock=none,addr=192.168.0.9) [root@CentOS-8 ~]# Problem not seem at kernel-4.18.0-147.8.1.el8_1.x86_64. Problem also seen at kernel-4.18.0-193.19.1.el8_2.x86_64 nfs-utils-1:2.3.3-26.el8.x86_64 Regards, Gordon | ||||
Tags | No tags attached. | ||||
Performed some additional testing: Built test system based off 8.2004 distribution, updated to latest kernel, hand mounted NFS server using defaults, ran the following test loop, which completed without any errors: [root@CentOS-8 ~]# for i in {1..2048} > do > dd if=/dev/urandom bs=1M of=/mnt/HomeNAS/Linux_Backups/CentOS8_Test/size_${i} count=${i} 2>&1 |grep -v record > rm -f /mnt/HomeNAS/Linux_Backups/CentOS8_Test/size_${i} > done Ran the same test on my original system and got failures at random file sizes. 153092096 bytes (153 MB, 146 MiB) copied, 2.31904 s, 66.0 MB/s 154140672 bytes (154 MB, 147 MiB) copied, 2.32882 s, 66.2 MB/s 155189248 bytes (155 MB, 148 MiB) copied, 2.30956 s, 67.2 MB/s 156237824 bytes (156 MB, 149 MiB) copied, 2.32989 s, 67.1 MB/s dd: closing output file '/mnt/HomeNAS/Linux_Backups/CentOS8_Test/size_150': Stale file handle 158334976 bytes (158 MB, 151 MiB) copied, 2.37034 s, 66.8 MB/s 159383552 bytes (159 MB, 152 MiB) copied, 2.36798 s, 67.3 MB/s 160432128 bytes (160 MB, 153 MiB) copied, 2.41636 s, 66.4 MB/s dd: closing output file '/mnt/HomeNAS/Linux_Backups/CentOS8_Test/size_154': Stale file handle 162529280 bytes (163 MB, 155 MiB) copied, 2.47603 s, 65.6 MB/s 163577856 bytes (164 MB, 156 MiB) copied, 2.44029 s, 67.0 MB/s 164626432 bytes (165 MB, 157 MiB) copied, 2.44239 s, 67.4 MB/s 165675008 bytes (166 MB, 158 MiB) copied, 2.46913 s, 67.1 MB/s 166723584 bytes (167 MB, 159 MiB) copied, 2.48472 s, 67.1 MB/s 167772160 bytes (168 MB, 160 MiB) copied, 2.50203 s, 67.1 MB/s 168820736 bytes (169 MB, 161 MiB) copied, 2.52402 s, 66.9 MB/s 169869312 bytes (170 MB, 162 MiB) copied, 2.52645 s, 67.2 MB/s 170917888 bytes (171 MB, 163 MiB) copied, 2.53589 s, 67.4 MB/s 171966464 bytes (172 MB, 164 MiB) copied, 2.56675 s, 67.0 MB/s 173015040 bytes (173 MB, 165 MiB) copied, 2.56817 s, 67.4 MB/s 174063616 bytes (174 MB, 166 MiB) copied, 2.60168 s, 66.9 MB/s 175112192 bytes (175 MB, 167 MiB) copied, 2.609 s, 67.1 MB/s 176160768 bytes (176 MB, 168 MiB) copied, 2.63066 s, 67.0 MB/s 177209344 bytes (177 MB, 169 MiB) copied, 2.64043 s, 67.1 MB/s dd: closing output file '/mnt/HomeNAS/Linux_Backups/CentOS8_Test/size_170': Stale file handle 179306496 bytes (179 MB, 171 MiB) copied, 2.68551 s, 66.8 MB/s 180355072 bytes (180 MB, 172 MiB) copied, 2.69241 s, 67.0 MB/s 181403648 bytes (181 MB, 173 MiB) copied, 2.70395 s, 67.1 MB/s 182452224 bytes (182 MB, 174 MiB) copied, 2.75667 s, 66.2 MB/s 183500800 bytes (184 MB, 175 MiB) copied, 2.72837 s, 67.3 MB/s 184549376 bytes (185 MB, 176 MiB) copied, 2.80426 s, 65.8 MB/s 185597952 bytes (186 MB, 177 MiB) copied, 2.77782 s, 66.8 MB/s 186646528 bytes (187 MB, 178 MiB) copied, 2.78408 s, 67.0 MB/s 187695104 bytes (188 MB, 179 MiB) copied, 2.80151 s, 67.0 MB/s 188743680 bytes (189 MB, 180 MiB) copied, 2.80926 s, 67.2 MB/s 189792256 bytes (190 MB, 181 MiB) copied, 2.87398 s, 66.0 MB/s 190840832 bytes (191 MB, 182 MiB) copied, 2.86521 s, 66.6 MB/s dd: closing output file '/mnt/HomeNAS/Linux_Backups/CentOS8_Test/size_183': Stale file handle 192937984 bytes (193 MB, 184 MiB) copied, 2.90914 s, 66.3 MB/s Stopped automounter and hand mounted NFS filesystem, again using the default mount option and the problem was not repeatable. Modified automounter file to remove non-default values and restarted autofs, ran test and observed no errors. automaount file: Linux_Backups -rw,credentials=/etc/synology_credentials.txt 192.168.0.9:/volume1/Linux Backups After a couple of iterations removing variables from the automount file I discovered that removing "wsize=8192" produces a working NFS mount Linux_Backups -rw,credentials=/etc/synology_credentials.txt,rsize=8192 192.168.0.9:/volume1/Linux Backups [root@CentOS-8 ~]# mount|grep Backup 192.168.0.9:/volume1/Linux Backups on /mnt/HomeNAS/Linux_Backups type nfs (rw,relatime,vers=3,rsize=8192,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.9,mountvers=3,mountport=892,mountproto=udp,local_lock=none,addr=192.168.0.9) [root@CentOS-8 ~]# So it looks like setting a wsize=8192 seems to be a good way to replicate the problem. |
|