NVME block-device-mapping symlinks are not created reliably

[nmeyerhans]

#9 changed several of the nvme udev rules to only trigger on on "add" events. However, this causes the links to be lost on "change" events, which occur whenever the corresponding device is opened in read/write mode (e.g. when creating or mounting a filesystem).

[lkinley-rythmos]

When will the fix hit AL2? This broke a lot of scripts for us as running pvcreate on the device symlink would cause it to disappear.

[nmeyerhans]

Hi @lkinley-rythmos Sorry for the inconvenience. I don't have specific ETA for the AL2 update. For the moment, I recommend either staying on version 1.2-46 1.2-45 of the package or pulling in the fixed 70-ec2-nvme-devices.rules from #23

[tyrken]

Hopefully you can get this update out soon - it broke a lot of our bootstrap scripts also.

FWIW in my testing I found 1.2-46.amzn2 the release that lost the nvme symlinks on formatting of attached EBS disks, our AMIs including 1.2-45.amzn2 were the ones to stick with as still working...

[cartermckinnon]

@nmeyerhans can you clarify this?

staying on version 1.2-46 of the package

It's my understanding from internal discussions that we should stay on 1.2-45, not 46. (i.e. this problem was introduced in 46). Is that correct?

[nmeyerhans]

Yes, that's correct. 1.2-46 is the one that introduced the problem. I've updated the earlier comment to correct that.

[nmeyerhans]

The 1.2-47.amzn2 packages are starting to show up in the Amazon Linux 2 repositories now:

[ec2-user@ip-10-0-0-102 ~]$ yum check-update
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
amzn2-core                                                                                                           | 3.7 kB  00:00:00     
(1/3): amzn2-core/2/x86_64/group_gz                                                                                  | 2.5 kB  00:00:00     
(2/3): amzn2-core/2/x86_64/updateinfo                                                                                | 447 kB  00:00:00     
(3/3): amzn2-core/2/x86_64/primary_db                                                                                |  60 MB  00:00:00     

aws-cfn-bootstrap.noarch                                          2.0-10.amzn2                                                    amzn2-core
ec2-utils.noarch                                                  1.2-47.amzn2                                                    amzn2-core
tzdata.noarch                                                     2021e-1.amzn2                                                   amzn2-core
vim-common.x86_64                                                 2:8.2.4314-1.amzn2.0.1                                          amzn2-core
vim-data.noarch                                                   2:8.2.4314-1.amzn2.0.1                                          amzn2-core
vim-enhanced.x86_64                                               2:8.2.4314-1.amzn2.0.1                                          amzn2-core
vim-filesystem.noarch                                             2:8.2.4314-1.amzn2.0.1                                          amzn2-core
vim-minimal.x86_64                                                2:8.2.4314-1.amzn2.0.1                                          amzn2-core

There may be some delay before it reaches every region, so please be patient if it's not available yet where you're running.

[dhs-rec]

@nmeyerhans, I see you're also the maintainer of the Debian/Ubuntu package (amazon-ec2-utils). I'm currently running into this issue on Ubuntu 22.04, which still ships 1.4 (w/o the fix). Any chance to get the Ubuntu package for 22.04 updated to 1.4.1 (in the near future)?

[nmeyerhans]

@dhs-rec I've uploaded 2.0.1 to Debian. I'm not at all involved in Ubuntu and have no idea what it takes to get it updated there.

[dhs-rec]

@nmeyerhans, thanks a lot. Maybe I need to open a ticket for Ubuntu, then...

[dhs-rec]

https://bugs.launchpad.net/ubuntu/+source/amazon-ec2-utils/+bug/2000857