Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource/aws_fsx_windows_file_system: Unexpected errors and panics with capacity updates #16440

Closed
bflad opened this issue Nov 25, 2020 · 4 comments · Fixed by #16488
Closed
Labels
bug Addresses a defect in current functionality. crash Results from or addresses a Terraform crash or kernel panic. service/fsx Issues and PRs that pertain to the fsx service.
Milestone

Comments

@bflad
Copy link
Contributor

bflad commented Nov 25, 2020

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform CLI v0.12.29

Terraform AWS Provider main branch

Affected Resource(s)

  • aws_fsx_windows_file_system

Terraform Configuration Files

variable "storage_capacity" {
  type = number
}

resource "aws_fsx_windows_file_system" "test" {
  active_directory_id = aws_directory_service_directory.test.id
  skip_final_backup   = true
  storage_capacity    = var.storage_capacity
  subnet_ids          = [aws_subnet.test1.id]
  throughput_capacity = 16
}

Debug Output

Unexpected error:

2020/11/24 17:55:04 [DEBUG] [aws-sdk-go] DEBUG: Response FSx/DescribeFileSystems Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Connection: close
Content-Length: 1250
Content-Type: application/x-amz-json-1.1
Date: Tue, 24 Nov 2020 17:55:04 GMT
X-Amzn-Requestid: e15748d3-3074-48a0-ad74-9aae0e00d4a1


-----------------------------------------------------
2020/11/24 17:55:04 [DEBUG] [aws-sdk-go] {"FileSystems":[{"AdministrativeActions":[{"AdministrativeActionType":"FILE_SYSTEM_UPDATE","RequestTime":1.606240473954E9,"Status":"PENDING","TargetFileSystemValues":{"StorageCapacity":36}},{"AdministrativeActionType":"STORAGE_OPTIMIZATION","RequestTime":1.606240473954E9,"Status":"PENDING"}],"CreationTime":1.606240455065E9,"DNSName":"fs-05f1b32969f95e5dc.corp.notexample.com","FileSystemId":"fs-05f1b32969f95e5dc","FileSystemType":"WINDOWS","KmsKeyId":"arn:aws:kms:us-west-2:--OMITTED--:key/21d2faae-2124-4cb5-9636-0159633a9f08","Lifecycle":"AVAILABLE","NetworkInterfaceIds":["eni-076385c73f83789e5"],"OwnerId":"187416307283","ResourceARN":"arn:aws:fsx:us-west-2:--OMITTED--:file-system/fs-05f1b32969f95e5dc","StorageCapacity":32,"StorageType":"SSD","SubnetIds":["subnet-09422bb0cd6ba51a7"],"Tags":[],"VpcId":"vpc-0644e8d2d1f523693","WindowsConfiguration":{"ActiveDirectoryId":"d-9267082361","AutomaticBackupRetentionDays":7,"CopyTagsToBackups":false,"DailyAutomaticBackupStartTime":"13:30","DeploymentType":"SINGLE_AZ_1","PreferredFileServerIp":"10.0.1.254","PreferredSubnetId":"subnet-09422bb0cd6ba51a7","RemoteAdministrationEndpoint":"fs-05f1b32969f95e5dc.corp.notexample.com","ThroughputCapacity":16,"WeeklyMaintenanceStartTime":"6:07:00"}}]}
2020/11/24 17:55:04 [WARN] Got error running Terraform: 2020/11/24 17:54:32 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0

Error: Error waiting for filesystem (fs-05f1b32969f95e5dc) to become available: unexpected state 'PENDING', wanted target 'COMPLETED, UPDATED_OPTIMIZING'. last error: %!s(<nil>)

Panic:

2020/11/24 17:50:12 [DEBUG] [aws-sdk-go] {"FileSystems":[{"CreationTime":1.606240165654E9,"DNSName":"amznfsxnll6tba2.corp.notexample.com","FileSystemId":"fs-0e21d63b1fa7770d9","FileSystemType":"WINDOWS","KmsKeyId":"arn:aws:kms:us-west-2:--OMITTED--:key/21d2faae-2124-4cb5-9636-0159633a9f08","Lifecycle":"UPDATING","NetworkInterfaceIds":["eni-0082d3d290934910d"],"OwnerId":"187416307283","ResourceARN":"arn:aws:fsx:us-west-2:--OMITTED--:file-system/fs-0e21d63b1fa7770d9","StorageCapacity":32,"StorageType":"SSD","SubnetIds":["subnet-08771b7bc84d10514"],"Tags":[],"VpcId":"vpc-03d8a668635aebc2a","WindowsConfiguration":{"AutomaticBackupRetentionDays":7,"CopyTagsToBackups":false,"DailyAutomaticBackupStartTime":"11:30","DeploymentType":"SINGLE_AZ_1","PreferredFileServerIp":"10.0.1.237","PreferredSubnetId":"subnet-08771b7bc84d10514","RemoteAdministrationEndpoint":"amznfsxnll6tba2.corp.notexample.com","SelfManagedActiveDirectoryConfiguration":{"DnsIps":["10.0.1.105","10.0.2.43"],"DomainName":"corp.notexample.com","FileSystemAdministratorsGroup":"Domain Admins","UserName":"Administrator"},"ThroughputCapacity":8,"WeeklyMaintenanceStartTime":"7:11:30"}}]}

Panic Output

=== CONT  TestAccAWSFsxWindowsFileSystem_SelfManagedActiveDirectory_Username
------- Stderr: -------
panic: runtime error: index out of range [0] with length 0
goroutine 5098 [running]:
github.com/terraform-providers/terraform-provider-aws/aws.refreshFsxFileSystemLifecycleOptimizing.func1(0xc00011def0, 0xc00011de48, 0x2, 0x1, 0x1, 0xc000512580)
  /opt/teamcity-agent/work/2e10e023da0c7520/src/github.com/terraform-providers/terraform-provider-aws/aws/fsx.go:67 +0x176
github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.(*StateChangeConf).WaitForStateContext.func1(0xc000b48180, 0xc0014b0230, 0xc0012eca20, 0xc0021a1840, 0xc0010cf150, 0xc0010cf148)
  /opt/teamcity-agent/work/2e10e023da0c7520/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.3.0/helper/resource/state.go:110 +0x329
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource.(*StateChangeConf).WaitForStateContext
  /opt/teamcity-agent/work/2e10e023da0c7520/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.3.0/helper/resource/state.go:83 +0x1c5

Other tests similarly panicked:

=== CONT  TestAccAWSFsxWindowsFileSystem_WeeklyMaintenanceStartTime
------- Stderr: -------
panic: runtime error: index out of range [0] with length 0

=== CONT  TestAccAWSFsxWindowsFileSystem_DailyAutomaticBackupStartTime
------- Stderr: -------
panic: runtime error: index out of range [0] with length 0

=== CONT  TestAccAWSFsxWindowsFileSystem_AutomaticBackupRetentionDays
------- Stderr: -------
panic: runtime error: index out of range [0] with length 0

Expected Behavior

PENDING status should be handled in waiter. No panics. 😄

Actual Behavior

PENDING status errors unexpectedly. Missing AdministrativeActions array panics.

Steps to Reproduce

  1. TF_ACC=1 go test -parallel=20 -run='TestAccAWSFsxWindowsFileSystem_' -timeout=4h -v

References

@bflad bflad added bug Addresses a defect in current functionality. crash Results from or addresses a Terraform crash or kernel panic. service/fsx Issues and PRs that pertain to the fsx service. labels Nov 25, 2020
@DrFaust92
Copy link
Collaborator

Possibly related to changes introduced at fsx with a new status: #16429

@bflad
Copy link
Contributor Author

bflad commented Nov 30, 2020

Draft PR for consideration: #16488

@bflad bflad removed their assignment Nov 30, 2020
@bflad bflad added this to the v3.19.0 milestone Dec 1, 2020
bflad added a commit that referenced this issue Dec 1, 2020
…ected errors, and use correct operation timeout on update (#16488)

Reference: https://docs.aws.amazon.com/fsx/latest/APIReference/API_AdministrativeAction.html
Reference: #16440

Output from acceptance testing:

```
Pending
```
@ghost
Copy link

ghost commented Dec 1, 2020

This has been released in version 3.19.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Dec 31, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked as resolved and limited conversation to collaborators Dec 31, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. crash Results from or addresses a Terraform crash or kernel panic. service/fsx Issues and PRs that pertain to the fsx service.
Projects
None yet
2 participants