 |
I have experienced this exact issue with 2 identical nvme disks, I am unable to remove any disks from the physical unit to proceed with installation and have instead had to pick Ubuntu. |
|
|
I can confirm this behaviour in both Centos version 7 and 8. I'm using the iso image and the same happens for both. I have two identical 500GB NVME drives and two identical 2TB NVME drives. The first two are identified correctly but the second two drives, the 2TB ones are being seen as a single multipath device. There's nothing wrong with the devices and they work fine on their own. So far I have not found a way around this but it's essential that I do. Will update if I find a solution. |
|
|
@draytoc I've found that in my case removing my first nvme (which hadls Windows installed) worked for me. Then all I had to do was simply update the grub boot config to refresh the boot images. |
|
|
Thanks @colinM9991 I need to configure both the 500GB drives and the 2TB drives each in mdraid so I would like to achieve this from the setup process. I'm using a SATA DOM for /boot. I've been trying to find out how to disable multipath from the kernel command line but to no avail. I think some distros allow for multipath=off but not sure about CentOS. |
|
|
@draytoc I tried 'inst.nompath' during the installation of my Fedora instance (this bug also exists on Fedora, presumably the same installer - Anaconda? My Linux knowledge is very poor) and had no luck. From the following docs, you may be able to use 'inst.nompath' to see if you have any success https://docs.centos.org/en-US/8-docs/standard-install/assembly_custom-boot-options/ |
|
|
Thanks again @colinM9991 I tried inst.nompath and it is completely ignored. Finally I tried adding dm-multipath.blacklist=1 to the kernel command line and this allowed me to see the NVME drives and set everything up. So the issue is resolved for me. |
|
|
This issue also exists as Bug 1845915 in Fedora's Bugzilla and as is the case here, there has been no reported progress since June 2020. Maybe I mistyped something, draytoc's blacklist method at the kernel boot up command did not work for me. This issue exists in the latest Fedora, Centos 8.2, Centos Stream - since all use the same Anaconda set up utility. My workaround, as some have noted, is to remove one of my two nvme drives. Luckily I had an SSD the same size as the remaining nvme which allowed me to put the nvme and the ssd in raid-1. After the OS is installed I manually fail the SSD and remove it. Power off the computer. Add the 2nd nvme. Reboot. OS comes up fine. mdadm reports the missing drive from the raid pair. After cloning the 1st nvme partitions to the 2nd nvme drive, I manually add the 2nd nvme to my raid-1 config and all is roses until my next OS re-install .... |
|
|
@bsandhu I'm surprised using dm-multipath.blacklist=1 didn't work for you. Were your NVME drives being seen as a single multipath device? I think this issue is specific to certain hardware. It doesn't happen for us in revision 2 of the system board we are using - with the same NVME drives. It might be specific to certain system boards and certain drives / NVME controller firmware. |
|
|
Yes, @draytoc. Two 0.5 TB nvme drives show up as one mpatha drive but only at 1/2 total capacity. A single nvme drive - shows up as a regular disk drive - no problem. Sticking with raiding with 1 nvme paired with an SSD then replacing the SSD with the 2nd nvme when done with Anaconda. Could be, as you said, due to the rev on my mobo X570 AORUS ELITE (rev. 1.0) even with the latest F30 - BIOS which has the updated AMD AGESA ComboV2 1.0.8.1. I have not encountered this issue with a 'deb' based distro that I also have in my environment. Having an interest in Centos Stream and after failing to install Centos Stream I ended up installing Centos 8.2 and then converting to Stream with 'dnf install centos-release-stream; dnf update -y. reboot. |
|
|
I can confirm I can replicate this issue in both Centos7 and Centos8, using NVME drives from different vendors as well (Kingston, and others). We also tried different Distros (Ubuntu, Debian, etc. ) and we can confirm the issue only occurs with CentOS. dm-multipath.blacklist=1 in the kernel boot options does resolve this issue for me as well, for all vendors. |
|
|
Also and also Fedora and Centos Stream, @TimondAU. In my opinion, ANACONDA needs an update on properly recognizing NVMe drives, which other distributions seem to have accommodated nicely in that NVMe drives show up as individual Disk drives instead of as Multipath drives. Leveraging the man page for 'lsscsi' and using my own server as an example, here is what I think needs fixing: lsscsi -d [N:0:1:1] disk HP SSD EX900 500GB__1 /dev/nvme0n1 [259:0] [N:1:1:1] disk HP SSD EX900 500GB__1 /dev/nvme1n1 [259:1] In the H:C:T:L tuple within square brackets on the left, for NVMe, drives, "N" appears in the 'H' (first ) position. The 'C' position for NVMe is the controller's Linux generated "char" device minor number which is the first number that appears in a typical NVMe controller name. On my server, note the "0" and "1" respectively: /dev/nvme0n1 and /dev/nvme1n1. The 'T' position for NVMe is the "CNTLID" value. The 'T' position for NVMe is the "CNTLID" value. The final 'L' position is the NVMe namespace identifier which is typically a sequential value starting at 1. The leading explicit "N" for NVMe devices is converted internally into a large value (32,767) that should not interfere with any Linux generated SCSI host number; it also means that the numeric sort used to show hosts (controllers) and devices (LUs or logical units) will always place NVMe devices and controllers after those that use the SCSI subsystem. Anaconda appears to be incorrectly parsing NVMe drives. |
- Anonymous
