Redeploying a resource group and child resources fails with "Error: Resource group was not found" when using a data resource within a module

[ghost]

This issue was originally opened by @derekwinters as hashicorp/terraform#23738. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.12.18
+ provider.azurerm v1.39.0

Terraform Configuration Files

# main.tf

provider "azurerm" {
  version         = "=1.39.0"
  subscription_id = "redacted"
}

data "azurerm_subnet" "subnet" {
  name                 = "redacted"
  virtual_network_name = "redacted"
  resource_group_name  = "resource_group_1"
}

# create.tf

resource "azurerm_resource_group" "resource_group" {
  name     = "resource_group_2"
  location = "eastus2"
}

# This section is from a module, but for the purpose of reproducing it's simplified and added directly to the create.tf file
data "azurerm_resource_group" "new_rg" {
  name = azurerm_resource_group.resource_group.name
}

resource "azurerm_virtual_machine" "vm" {
  name                             = "test_vm"
  location                         = data.azurerm_resource_group.new_rg.location
  resource_group_name              = data.azurerm_resource_group.new_rg.name
  vm_size                          = "Standard_B1ls"
  delete_os_disk_on_termination    = true
  delete_data_disks_on_termination = true
  network_interface_ids            = [azurerm_network_interface.nic1.id]

  storage_os_disk {
    name          = "test_vm_osdisk"
    caching       = "ReadWrite"
    create_option = "FromImage"
  }

  storage_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "18.04-LTS"
    version   = "latest"
  }

  os_profile {
    computer_name  = "testvm"
    admin_username = "myadmin"
    admin_password = "TestP@ss1!"
  }

  os_profile_linux_config {
    disable_password_authentication = false
  }

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_network_interface" "nic1" {
  name                = "test_vm_nic"
  location            = data.azurerm_resource_group.new_rg.location
  resource_group_name = data.azurerm_resource_group.new_rg.name

  ip_configuration {
    name                          = "test_vm_ip"
    subnet_id                     = data.azurerm_subnet.subnet.id
    private_ip_address_allocation = "Dynamic"
  }
}

Expected Behavior

If this configuration is deployed and needs to be redeployed, sometimes it is easier to remove the configuration (instead of terraform taint), then add the configuration back to redeploy. It would be expected to redeploy successfully.

Actual Behavior

If a configuration is removed and applied, and then added back and applied, the data "azurerm_resource_group" "new_rg" resource will fail with Error: Error: Resource Group "resource_group_2" was not found

Steps to Reproduce

1. terraform init
2. terraform apply
3. mv create.tf create.tf.bak
4. terraform apply
5. mv create.tf.bak create.tf
6. terraform apply

Additional Context

If there is anything in the configuration after step 4, the error occurs. In this example, if the data "azurerm_subnet" data resource is also removed, the error does not occur.

I've found two ways to work around this bug

1. Remove everything else from the configuration and terraform apply, then add everything back. This obviously isn't ideal, but it does work in the example if I also remove the data "azurerm_subnet" data resource and terraform apply, then add everything back, terraform apply will work again. If I remove the subnet data, terraform apply, add the subnet data back and terraform apply again, then add all the resources back, the error re-occurs.
2. Add only the new_rg resource back, terraform apply, then add the rest of the resources that go in that resource group.

[derekwinters]

I did some more testing and you actually don't need to create any resources. Simply creating the resource group and having a data resource with the output from the resource group creation is enough to cause this issue to happen.

# create.tf
resource "azurerm_resource_group" "resource_group" {
  name     = "resource_group_2"
  location = "eastus2"
}

data "azurerm_resource_group" "new_rg" {
  name = azurerm_resource_group.resource_group.name
}

1. terraform init
2. terraform apply
3. mv create.tf create.tf.bak
4. terraform apply
5. mv create.tf.bak create.tf
6. terraform apply

[Gvazzana]

Has there been any progress on this issue?

I am seeing the exact same behavior when attempting to use the resource group data source, in Azurerm provider 1.39 (tf version .012)

data "azurerm_resource_group" "core-rg" {
name = "${var.project_ident}-${var.env_ident}-${var.core_rg_name}"
}

I get "Error: Error: Resource Group "rg_name" was not found"

And if i try to use the rg data source, as a reference for another data source, i get both data sources dont exist

data "azurerm_key_vault" "kv" {
name = "${var.project_ident}-${var.env_ident}-${var.kv_name}"
resource_group_name = data.azurerm_resource_group.core-rg.name
}

I get "Error: KeyVault "kv_name" (Resource Group "rg_name") does not exist"

This is prohibiting me from using my modules now that i have upgraded to .012 to take advantage of new features

[vfab]

#Bump
I am observing the exact same behavior after upgrading to TF 0.12 for both the data providers mentioned in @Gvazzana's post.
Please note that this is a blocker that prevents creating reusable modules that try to reference existing Resource Groups or Key Vaults.

[colin-lyman]

Same issue when re-running a plan/apply when a VM has been "deleted" via the console. cannot rerun which should re-add the VM.

[idokaplan]

Hi,

Same issue when re-running a plan/apply when a VM has been "deleted" via the console. cannot rerun which should re-add the VM.

Is there a workaround for this issue? ETA for fix?

Please advise.
Thanks,
Ido

[pbstrein]

The same issue appears when using a subnet.

[thos25]

This still occurs. I have a similar config where i deploy a number of VMs. The initial apply works, but if I increase the number of VM's after the fact, it fails stating the resource group was not found.

[msygmatic]

+1

I'm hitting a similar issue for ANYTHING that needs a Resource Group and I happen to deploy a RG alongside any resources that will be inside said RG.

One day... one day we will finally finally get depends_on support for modules (which I am using in my case).

[tombuildsstuff]

👋

Taking a look through here this appears to be an issue regarding Module Dependencies - which unfortunately aren't supported in Terraform Core at this time. As such I'm going to close this in favour of the upstream issue on Terraform Core where this is being tracked: hashicorp/terraform#10462 - would you mind subscribing to that issue for updates?

Thanks!

[colin-lyman]

@tombuildsstuff I'm not sure why you think this is about Module dependencies, please see the first example. modules aren't being used. In my use case just deleting a VM on the console has the same problem.

[derekwinters]

@tombuildsstuff The original example is from a module, but it isn't trying to use the explicit depends_on parameter.

This works the first time it's used, but a redeploy is what causes the inconsistent plan.

[schlbra]

@tombuildsstuff or @katbyte , Can this issue please be re-opened as it is not an issue regarding Module dependencies? We would greatly appreciate it as this issue resurfaces in our environment at least once a week in our pipelines.

[HighwayofLife]

Running into this problem it's very confusing why this is an issue. Unfortunately, they closed the ticket as "too heated"!!! I'm capturing a simple use case here for... posterity, I suppose.

app/main.tf

resource "azurerm_resource_group" "group" {
  name = "MyRG"
  location = "westus"
}

module "child" {
  source = "../modules/rg-ref"
  resource_group_name = azurerm_resource_group.group.name
}

modules/rg-ref/main.tf

variable "resource_group_name" {}

data "azurerm_resouce_group" "group" {
  name = var.resource_group_name
}

Error: Error: Resource Group "ShipyardStorage" was not found

  on ../../modules/reg-ref/main.tf line 3, in data "azurerm_resource_group" "group":
   1: data "azurerm_resource_group" "group" {

This makes no sense since we're explicitly declaring the resource as a dependent variable to the module, even though we don't have the strict depends_on functionality.

[umashankar2]

I am seeing the same issue with Terraform v0.12.24

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!