Racy behavior creating aws_routes that are inserted before other routes

[hashibot]

This issue was originally opened by @erickt as hashicorp/terraform#11811. It was migrated here as part of the provider split. The original body of the issue is below.

---

I'm encountering some racy behavior creating an aws_routes. It appears that when a new route is created, and it is inserted before other routes, terraform doesn't consistently handle the destruction and creation of the routes correctly. This can result in breaking routing in a VPC, which requires manual intervention in fixing the routes in the console and manually updating the terraform.tfstate file.

Terraform Version

Terraform v0.8.6

Affected Resource(s)

• aws_route

Terraform Configuration Files

I ended up writing some python programs to reproduce it since it doesn't happen consistently.

Debug Output

Logs are in this gist.

Expected Behavior

The routes created.

Actual Behavior

One route will occasionally not be created, or terraform won't add the created routes to the terraform.tfstate file.

Steps to Reproduce

For the racy creation with aws_internet_gateway:

1. Run AWS_PROFILE=... ./run-igw.py --vpc-cidr 10.0.0.0/16 --availability-zone us-west-2a --subnet-cidr 10.0.0.0/20 10, which will create and remove routes in a loop, where 10 is the cycle.
2. After a number of cycles, this should eventually error out with:

* aws_route.private_peer_route.1: Error finding route after creating it: Unable to find matching route for Route Table (rtb-23e3d744) and destination CIDR block (12.0.0.0/32).

For the racy creation with VPC peers:

1. Run AWS_PROFILE=... ./run-peer.py --vpc-cidr 10.0.0.0/16 --availability-zone us-west-2a --subnet-cidr 10.0.0.0/20 --peers pcx-d20181bb=11.0.0.0/32,pcx-aa0181c3=12.0.0.0/32 10, which will create and remove routes in a loop, where 10 is the cycle. The peer cidrs don't matter.
2. After a number of cycles, this should eventually error out with:

* aws_route.private_peer_route.1: Error creating route: RouteAlreadyExists: The route identified by 11.0.0.0/32 already exists.
	status code: 400, request id: ccb35b15-6609-401e-8b0d-b9ce14f2f510

References

Perhaps this just need locks aws_security_group_rule like in 6b6b5a43c32f7e6bc4b90db79d44494c72398819?

[afritzler]

Experiencing the same problem here. Any ideas how to solve that? (using Terraform v0.11.7)

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!