Terraform aws provider destroy removes ec2 instance from local state without comparing the region

[Alex2357]

I have created an EC2 instance in us-west-2. Then I run terraform destroy providing an incorrect/wrong region us-west-1 it removes the EC2 instance from local state. while it was in the state kept as in the us-west-2 region.
So effectively state is no longer tracking that EC2 instance which was created in us-west-2.

Terraform CLI and Terraform AWS Provider Version

john@john-VirtualBox:~/destroybug$ terraform -v
Terraform v0.13.3
+ provider registry.terraform.io/hashicorp/aws v3.7.0
john@john-VirtualBox:~/destroybug$ aws --version
aws-cli/2.0.50 Python/3.7.3 Linux/5.4.0-47-generic exe/x86_64.ubuntu.18
john@john-VirtualBox:~/destroybug$ 

Affected Resource(s)

• aws_3.7

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
    }
  }
}

variable "region" {
    
}

provider "aws" {
  profile = "default"
  region  = var.region
}

resource "aws_instance" "example" {
  ami           = "ami-08d70e59c07c61a3a"
  instance_type = "t2.micro"
}

Debug Output

john@john-VirtualBox:~/destroybug$ # Display What we have in local file system & in AWS
john@john-VirtualBox:~/destroybug$ ls -la
total 12
drwxr-xr-x  2 john john 4096 Sep 22 09:14 .
drwxr-xr-x 22 john john 4096 Sep 22 08:33 ..
-rw-r--r--  1 john john  293 Sep 22 09:08 destroybug.tf
john@john-VirtualBox:~/destroybug$ cat destroybug.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
    }
  }
}

variable "region" {
    
}

provider "aws" {
  profile = "default"
  region  = var.region
}

resource "aws_instance" "example" {
  ami           = "ami-08d70e59c07c61a3a"
  instance_type = "t2.micro"
}john@john-VirtualBox:~/destroybug$ aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilitZone, State.Name, InstanceId]' --output text --region us-west-2
us-west-2c      terminated      i-0e501a9246eadb6e0
john@john-VirtualBox:~/destroybug$ 
john@john-VirtualBox:~/destroybug$ 
john@john-VirtualBox:~/destroybug$ 
john@john-VirtualBox:~/destroybug$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Installing hashicorp/aws v3.7.0...
- Installed hashicorp/aws v3.7.0 (signed by HashiCorp)

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.

* hashicorp/aws: version = "~> 3.7.0"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
john@john-VirtualBox:~/destroybug$ terraform apply -var 'region=us-west-2' -auto-approve
aws_instance.example: Creating...
aws_instance.example: Still creating... [10s elapsed]
aws_instance.example: Still creating... [20s elapsed]
aws_instance.example: Still creating... [30s elapsed]
aws_instance.example: Still creating... [40s elapsed]
aws_instance.example: Creation complete after 41s [id=i-009137c590d4c4e18]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
john@john-VirtualBox:~/destroybug$ terraform show
# aws_instance.example:
resource "aws_instance" "example" {
    ami                          = "ami-08d70e59c07c61a3a"
    arn                          = "arn:aws:ec2:us-west-2:593925197038:instance/i-009137c590d4c4e18"
    associate_public_ip_address  = true
    availability_zone            = "us-west-2c"
    cpu_core_count               = 1
    cpu_threads_per_core         = 1
    disable_api_termination      = false
    ebs_optimized                = false
    get_password_data            = false
    hibernation                  = false
    id                           = "i-009137c590d4c4e18"
    instance_state               = "running"
    instance_type                = "t2.micro"
    ipv6_address_count           = 0
    ipv6_addresses               = []
    monitoring                   = false
    primary_network_interface_id = "eni-09c309d81d000b029"
    private_dns                  = "ip-172-31-12-204.us-west-2.compute.internal"
    private_ip                   = "172.31.12.204"
    public_dns                   = "ec2-18-237-197-150.us-west-2.compute.amazonaws.com"
    public_ip                    = "18.237.197.150"
    secondary_private_ips        = []
    security_groups              = [
        "default",
    ]
    source_dest_check            = true
    subnet_id                    = "subnet-13f5584e"
    tenancy                      = "default"
    volume_tags                  = {}
    vpc_security_group_ids       = [
        "sg-04be092e",
    ]

    credit_specification {
        cpu_credits = "standard"
    }

    metadata_options {
        http_endpoint               = "enabled"
        http_put_response_hop_limit = 1
        http_tokens                 = "optional"
    }

    root_block_device {
        delete_on_termination = true
        device_name           = "/dev/sda1"
        encrypted             = false
        iops                  = 100
        volume_id             = "vol-07b0137812a18ad80"
        volume_size           = 8
        volume_type           = "gp2"
    }
}
john@john-VirtualBox:~/destroybug$ cp terraform.tfstate terraform.tfstate.backup-justcreated
john@john-VirtualBox:~/destroybug$ terraform destroy -var 'region=us-west-1' -force
aws_instance.example: Refreshing state... [id=i-009137c590d4c4e18]

Destroy complete! Resources: 0 destroyed.
john@john-VirtualBox:~/destroybug$ terraform show

john@john-VirtualBox:~/destroybug$ cat terraform.tfstate
{
  "version": 4,
  "terraform_version": "0.13.3",
  "serial": 2,
  "lineage": "55a9dfc1-df93-d127-2e43-3fdd31ec5b20",
  "outputs": {},
  "resources": []
}
john@john-VirtualBox:~/destroybug$ 
john@john-VirtualBox:~/destroybug$ # check the AWS instance still there as it should, but not in the state!!!
john@john-VirtualBox:~/destroybug$ aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilityZone, State.Name, InstanceId]' --output text --region us-west-2
us-west-2c      terminated      i-0e501a9246eadb6e0
us-west-2c      running i-009137c590d4c4e18
john@john-VirtualBox:~/destroybug$ cp terraform.tfstate.backup-justcreated terraform.tfstate 
john@john-VirtualBox:~/destroybug$ terraform show
# aws_instance.example:
resource "aws_instance" "example" {
    ami                          = "ami-08d70e59c07c61a3a"
    arn                          = "arn:aws:ec2:us-west-2:593925197038:instance/i-009137c590d4c4e18"
    associate_public_ip_address  = true
    availability_zone            = "us-west-2c"
    cpu_core_count               = 1
    cpu_threads_per_core         = 1
    disable_api_termination      = false
    ebs_optimized                = false
    get_password_data            = false
    hibernation                  = false
    id                           = "i-009137c590d4c4e18"
    instance_state               = "running"
    instance_type                = "t2.micro"
    ipv6_address_count           = 0
    ipv6_addresses               = []
    monitoring                   = false
    primary_network_interface_id = "eni-09c309d81d000b029"
    private_dns                  = "ip-172-31-12-204.us-west-2.compute.internal"
    private_ip                   = "172.31.12.204"
    public_dns                   = "ec2-18-237-197-150.us-west-2.compute.amazonaws.com"
    public_ip                    = "18.237.197.150"
    secondary_private_ips        = []
    security_groups              = [
        "default",
    ]
    source_dest_check            = true
    subnet_id                    = "subnet-13f5584e"
    tenancy                      = "default"
    volume_tags                  = {}
    vpc_security_group_ids       = [
        "sg-04be092e",
    ]

    credit_specification {
        cpu_credits = "standard"
    }

    metadata_options {
        http_endpoint               = "enabled"
        http_put_response_hop_limit = 1
        http_tokens                 = "optional"
    }

    root_block_device {
        delete_on_termination = true
        device_name           = "/dev/sda1"
        encrypted             = false
        iops                  = 100
        volume_id             = "vol-07b0137812a18ad80"
        volume_size           = 8
        volume_type           = "gp2"
    }
}
john@john-VirtualBox:~/destroybug$ 
john@john-VirtualBox:~/destroybug$ 
john@john-VirtualBox:~/destroybug$ 
john@john-VirtualBox:~/destroybug$ 
john@john-VirtualBox:~/destroybug$ aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilityZone, State.Name, InstanceId]' --output text --region us-west-2
us-west-2c      terminated      i-0e501a9246eadb6e0
us-west-2c      running i-009137c590d4c4e18
john@john-VirtualBox:~/destroybug$ terraform destroy -var 'region=us-west-2' -force
aws_instance.example: Refreshing state... [id=i-009137c590d4c4e18]
aws_instance.example: Destroying... [id=i-009137c590d4c4e18]
aws_instance.example: Still destroying... [id=i-009137c590d4c4e18, 10s elapsed]
aws_instance.example: Still destroying... [id=i-009137c590d4c4e18, 20s elapsed]
aws_instance.example: Still destroying... [id=i-009137c590d4c4e18, 30s elapsed]
aws_instance.example: Destruction complete after 33s

Destroy complete! Resources: 1 destroyed.
john@john-VirtualBox:~/destroybug$ aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilityZone, State.Name, InstanceId]' --output text --region us-west-2
us-west-2c      terminated      i-0e501a9246eadb6e0
us-west-2c      terminated      i-009137c590d4c4e18
john@john-VirtualBox:~/destroybug$ 

Expected Behavior

The EC2 instance should remain in the local state after destroy for wrong region is executed

Actual Behavior

The EC2 instance was removed from the state

Steps to Reproduce

Here's the destroybug.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
    }
  }
}

variable "region" {
    
}

provider "aws" {
  profile = "default"
  region  = var.region
}

resource "aws_instance" "example" {
  ami           = "ami-08d70e59c07c61a3a"
  instance_type = "t2.micro"
}

Here's the list of commands I executed the important moment is that instance is not displayed when execute terraform show \cat terraform.tfstate, while it should as I believe.

# Display What we have in local file system & in AWS
ls -la
cat destroybug.tf
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilityZone, State.Name, InstanceId]' --output text --region us-west-2


terraform init
terraform apply -var 'region=us-west-2' -auto-approve
terraform show
cp terraform.tfstate terraform.tfstate.backup-justcreated
terraform destroy -var 'region=us-west-1' -force
terraform show
cat terraform.tfstate

# check the AWS instance still there as it should, but not in the state!!!
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilityZone, State.Name, InstanceId]' --output text --region us-west-2
cp terraform.tfstate.backup-justcreated terraform.tfstate 
terraform show


aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilityZone, State.Name, InstanceId]' --output text --region us-west-2
terraform destroy -var 'region=us-west-2' -force
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Placement.AvailabilityZone, State.Name, InstanceId]' --output text --region us-west-2

[gdavison]

Hi @Alex2357, thanks for raising this. For a more detailed discussion on this issue, please take a look at #15052.

We understand that the current behaviour doesn't match the expectations of many users of Terraform. One workaround is to not pass the region as a parameter for each Terraform invocation. This can be done using build tooling, whether it's build scripts or Terraform Cloud, etc. A comment on the issue above gives more details: #15052 (comment).

One additional option is to always run terraform plan before apply to double-check that only the changes you expect will be applied. terraform apply will also show the plan as long as the -auto-approve flag is not set.

Since we're already tracking this problem in #15052, I'm going to close this issue. We do track "popularity" of issues, so you can add a 👍 reaction to that issue if you'd like.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!