Cross Subscription Image Builds Failing - newer resource-manager version solves issue

[jercle]

Overview of the Issue

Until the last week or so, we have been building images in one subscription then saving them in a Shared Image Gallery in another subscription. This is now failing with the following error:

2024/04/10 03:58:48 ui error: --> azure-arm.test1: performing CreateOrUpdate: unexpected status 400 (400 Bad Request) 
with error: InvalidParameter: The source VM '/subscriptions/PKR_VAR_SUB_ID/resourceGroups/pkr-Resource-Group-
s7mcoo7lju/providers/Microsoft.Compute/virtualMachines/pkrvms7mcoo7lju' is a cross subscription source for the 
galleryImageVersion created in subscription 'SIG_SUB_ID'. Please retry the call by specifying the VM id in 
'galleryImageVersion.properties.storageProfile.source.virtualMachineId' instead. See https://aka.ms/acgsecurityupdates
for more details.

Reproduction Steps

Run any packer build that builds an image, then saves the image to a SIG in another subscription

Plugin and Packer version

Packer v1.10.1
Failed on both versions of azure plugin. Previously functional on v2.0.2, only installed v2.1.0 after failures began.
packer-plugin-azure_v2.0.2_x5.0_linux_amd64
packer-plugin-azure_v2.1.0_x5.0_linux_amd64

Simplified Packer Buildfile

variable "CLIENT_ID" { type = string }

variable "CLIENT_SECRET" {
type = string
}

variable "SUBSCRIPTION_ID" {
type = string
}
variable "TENANT_ID" {
type = string
}

source "azure-arm" "test1" {
azure_tags = {
dept = "Engineering"
task = "Image deployment"
}

location = "Australia East"

virtual_network_name = "PKR_VAR_SUB_VNET"
virtual_network_subnet_name = "PKR_VAR_SUB_SUBNET"
virtual_network_resource_group_name = "PKR_VAR_SUB_RG"

os_type = "Windows"
image_offer = "office-365"
image_publisher = "MicrosoftWindowsDesktop"
image_sku = "win10-22h2-avd-m365-g2"

shared_image_gallery_timeout = "2h5m2s"
client_id = var.CLIENT_ID
client_secret = var.CLIENT_SECRET
communicator = "winrm"

shared_image_gallery_destination {
subscription = "SIG_SUB_ID"
resource_group = "SIG_RG"
gallery_name = "SIG_NAME"
image_name = "imgdef-specialised-win10-multi-session-gen2"
image_version = "1.0.89"
specialized = true
# replication_regions = ["australiaeast"]
}

user_assigned_managed_identities = [
"/subscriptions/SIG_SUB_ID/resourceGroups/PKR_VAR_SUB_RG/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id"
]

os_disk_size_gb = 127
subscription_id = var.SUBSCRIPTION_ID
tenant_id = var.TENANT_ID
vm_size = "Standard_F8s_V2"

winrm_insecure = true
winrm_timeout = "2h"
winrm_use_ssl = false
winrm_username = "packer"
winrm_password = ""
winrm_no_proxy = true

polling_duration_timeout = "1h5m2s"

custom_script = "winrm quickconfig -quiet"

custom_script = "powershell -ExecutionPolicy Unrestricted -NoProfile -NonInteractive -Command "$userData = (Invoke-RestMethod -Headers @{Metadata=$true} -Method GET -Uri http://IP/metadata/instance/compute/userData?api-version=2021-01-01$([char]38)format=text); $contents = [System.Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($userData)); set-content -path c:\Windows\Temp\userData1.ps1 -value $contents; . c:\Windows\Temp\userData1.ps1;""
user_data_file = "./userData1.ps1"

}

build {
sources = ["source.azure-arm.test1"]

provisioner "powershell" {
script = "./build.ps1"
elevated_user = "packer"
elevated_password = build.Password
execution_policy = "bypass"
}
}

Operating system and Environment details

Ubuntu x86-64 22.04

[jercle]

Could be related to Azure/azure-cli#28700 (Azure/azure-cli#28700 (comment)) or an issue with Azure's underlying API

[jercle]

Also, there's a new field in the GalleryArtifactVersionFullSource struct which should solve the issue by updating the used sdk and passing the value there

azure-sdk-for-go changelog

5.6.0 (2024-03-22)

Features Added

• New field VirtualMachineID in struct GalleryArtifactVersionFullSource

[jercle]

Also looks to be available in resource-manager/compute/2023-07-03

Having a look through the tags, the earliest available version of github.com/hashicorp/go-azure-sdk/resource-manager is v0.20240403.1161138

[jercle]

Pinning at least v0.20240403.1161138 of github.com/hashicorp/go-azure-sdk/resource-manager and VirtualMachineID here should resolve this issue

[JenGoldstrich]

Ah yeah, seems like we need to send the new VirtualMachineID field instead of ID for that case, but I think we still need to send ID if the build is sourced from a managed image and not a VM, I'll work on getting a fix out for that shortly, thanks for reporting and your investigation