NIC not disassociated before deletion

[gaui]

Overview of the Issue

When running Packer with the Azure plugin, it doesn't disassociate the NIC before deleting the VM, so it crashes (see log below).

Reproduction Steps

Run Packer to create an Azure VM image. The buildfile is here: https://github.com/actions/runner-images/blob/main/images/linux/ubuntu2004.json

Plugin and Packer version

Packer: 1.8.5
Packer Azure plugin: 1.3.1

Simplified Packer Buildfile

https://github.com/actions/runner-images/blob/main/images/linux/ubuntu2004.json

Operating system and Environment details

The Packer process runs on ubuntu-latest in an Azure DevOps Pipeline agent. The VM image Packer is creating is based on Ubuntu 20.04 LTS (https://github.com/actions/runner-images)

Log Fragments and crash.log files

==> azure-arm: Adding to deletion queue -> Microsoft.Network/networkInterfaces : 'pkrniho2hi13cqd'
==> azure-arm: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniho2hi13cqd'
==> azure-arm: Waiting for deletion of all resources...
==> azure-arm: Attempting deletion -> Microsoft.Compute/virtualMachines : 'pkrvmho2hi13cqd'
==> azure-arm: Error deleting resource. Will retry.
==> azure-arm: Name: pkrniho2hi13cqd
==> azure-arm: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Network/networkInterfaces/pkrniho2hi13cqd is used by existing resource /subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Compute/virtualMachines/pkrvmho2hi13cqd. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm:
==> azure-arm: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniho2hi13cqd'
==> azure-arm: Error deleting resource. Will retry.
==> azure-arm: Name: pkrniho2hi13cqd
==> azure-arm: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Network/networkInterfaces/pkrniho2hi13cqd is used by existing resource /subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Compute/virtualMachines/pkrvmho2hi13cqd. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm:
==> azure-arm: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniho2hi13cqd'
==> azure-arm: Error deleting resource. Will retry.
==> azure-arm: Name: pkrniho2hi13cqd
==> azure-arm: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Network/networkInterfaces/pkrniho2hi13cqd is used by existing resource /subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Compute/virtualMachines/pkrvmho2hi13cqd. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm:
==> azure-arm: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniho2hi13cqd'
==> azure-arm:  Deleting -> Microsoft.Compute/disks : '/subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Compute/disks/pkrosho2hi13cqd'
==> azure-arm: Removing the created Deployment object: 'pkrdpho2hi13cqd'
==> azure-arm: 
==> azure-arm: The resource group was not created by Packer, not deleting ...

[packardG]

I also face the same issue running packer version 1.9.4. Not only is the NIC not being disassociated/deleted, but the public IP & virtual network see that same behavior. The image builds don't fail but this does mean my RG is left with lots of stale resources that I need to clean up.

==> azure-arm.centos: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrnijsuvd7yb6f'                                                                                                                                          
==> azure-arm.centos: Attempting deletion -> Microsoft.Network/publicIPAddresses : 'pkripjsuvd7yb6f'                                                                                                                                          
==> azure-arm.centos: Attempting deletion -> Microsoft.Compute/virtualMachines : 'pkrvmjsuvd7yb6f'                                                                                                                                            
==> azure-arm.centos: Attempting deletion -> Microsoft.Network/virtualNetworks : 'pkrvnjsuvd7yb6f'                                                                                                                                            
==> azure-arm.centos: Error deleting resource. Will retry.                                                                                                                                                                                    
==> azure-arm.centos: Name: pkrnijsuvd7yb6f                                                                            
==> azure-arm.centos: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/.../resourceGroups/.../providers/Microsoft.Network/networkInterfaces/pkrnijsuvd7yb6f is used by existing resource /subscriptions/.../resourceGroups/.../providers/Microsoft.Compute/virtualMachines/pkrvmjsuvd7yb6f. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]                                                                                                                  
==> azure-arm.centos:                                                                                                  
==> azure-arm.centos: Error deleting resource. Will retry.                                                             
==> azure-arm.centos: Name: pkripjsuvd7yb6f                                                                            
==> azure-arm.centos: Error: network.PublicIPAddressesClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="PublicIPAddressCannotBeDeleted" Message="Public IP address /subscriptions/.../resourceGroups/.../providers/Microsoft.Network/publicIPAddresses/pkripjsuvd7yb6f can not be deleted since it is still allocated to resource /subscriptions/.../resourceGroups/.../providers/Microsoft.Network/networkInterfaces/pkrnijsuvd7yb6f/ipConfigurations/ipconfig. In order to delete the public IP, disassociate/detach the Public IP address from the resource.  To learn how to do this, see aka.ms/deletepublicip." Details=[]
==> azure-arm.centos:                                                                                                  
==> azure-arm.centos: Error deleting resource. Will retry.                                                             
==> azure-arm.centos: Name: pkrvnjsuvd7yb6f                                                                            
==> azure-arm.centos: Error: network.VirtualNetworksClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="InUseSubnetCannotBeDeleted" Message="Subnet pkrsnjsuvd7yb6f is in use by /subscriptions/.../resourceGroups/.../providers/Microsoft.Network/networkInterfaces/PKRNIJSUVD7YB6F/ipConfigurations/IPCONFIG and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet." Details=[]

[mfortin]

Thi is still happening with packer-plugin-azure_v2.0.2_x5.0_linux_amd64

[JenGoldstrich]

Hey folks, just wanted to chime in here since I see the activity going back to last year, sorry for the delay in responding to this issue.

The issue is exactly as the title says "NIC not disassociated before deletion", we add several deployment resources to a queue and retry if they fail until they were successfully deleted, I believe this was done to ensure that no resources were left behind. There's a few reports in this thread of orphaned resources and failed builds, but as far as I can tell the issue with the errors that looked like the below quoted text did not cause either of these, when a resource was actually failed for the final retry it sends a log mentioning that the resource should be manually deleted, and not "will retry".

Looking at @gaui's output from last year

==> azure-arm:
==> azure-arm: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniho2hi13cqd'
==> azure-arm: Error deleting resource. Will retry.
==> azure-arm: Name: pkrniho2hi13cqd
==> azure-arm: Error: network.InterfacesClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="NicInUse" Message="Network Interface /subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Network/networkInterfaces/pkrniho2hi13cqd is used by existing resource /subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Compute/virtualMachines/pkrvmho2hi13cqd. In order to delete the network interface, it must be dissociated from the resource. To learn more, see aka.ms/deletenic." Details=[]
==> azure-arm:
==> azure-arm: Attempting deletion -> Microsoft.Network/networkInterfaces : 'pkrniho2hi13cqd'
==> azure-arm:  Deleting -> Microsoft.Compute/disks : '/subscriptions/[ID]/resourceGroups/[RG]/providers/Microsoft.Compute/disks/pkrosho2hi13cqd'
==> azure-arm: Removing the created Deployment object: 'pkrdpho2hi13cqd'
==> azure-arm: 
==> azure-arm: The resource group was not created by Packer, not deleting ...

You can see that we do another attempt to delete the NIC after deleting the VM failed, but that we do another attempt which does not report failure, its a bit confusing but this means that the deletion was eventually successful after some retrying.

I understand this output is still not ideal even if the result is still a deleted network interface, as it creates confusion, looking at the output today its hard to understand when a resource has been successfully deleted, only when it has failed to be deleted, which makes it harder for users to parse what is happening

I have opened a PR #410 to address this issue by always deleting the build virtual machine first, and then the NIC, and then adding the other resources to a retry queue, this should eliminate the extraneous errors, if you still see errors like this after the above PR is merged and a new release is created, please open a new PR with the full output of your packer build, including the end of it, so we can confirm if builds are failing or crashing. If we're seeing orphaned resources as has been reported in this thread it is definitely an issue we'd want to investigate, but I think it is a separate issue from the NIC failing to delete and retrying until it eventually succeeds.