Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FSx File Gateway Support to Storage Gateway & Add FSx Association resource #20082

Merged
merged 54 commits into from
Jul 27, 2021
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
cf6e5e3
create fsx gateway & tests
drewmullen Jun 4, 2021
122de90
add tag tests
drewmullen Jul 6, 2021
aa5c5b9
docs
drewmullen Jul 13, 2021
905cd4d
fix lint errors
drewmullen Jul 14, 2021
1e56ec5
SG-FSx include AuditArn if empty string.
AdamTylerLynch Jul 14, 2021
df4fb0e
SG-FSx extract the lookup logic to the relevant finder package.
AdamTylerLynch Jul 14, 2021
36bcc50
SG-FSx added AccTest step to unset audit arn
AdamTylerLynch Jul 14, 2021
d2245fb
SG-FSx updated formatting in the docs
AdamTylerLynch Jul 14, 2021
74140c2
SG-FSx added audit disabled AccTest and fixed formating issues.
AdamTylerLynch Jul 14, 2021
ab79787
SG-FSx modified AccTest for AuditArn
AdamTylerLynch Jul 14, 2021
c6489d6
SG-FSx updated the state change for update
AdamTylerLynch Jul 14, 2021
7c7fd86
GS-FSx modified waiter/stats to use new finder method
AdamTylerLynch Jul 15, 2021
69a2a90
add tests for dependencies disappearing
drewmullen Jul 16, 2021
05c23c2
fixed doc errors
drewmullen Jul 19, 2021
c907aa4
use contants for fsx pending / target states
drewmullen Jul 20, 2021
83bc56b
add sgw arn to err, rm unnecessary setid()
drewmullen Jul 20, 2021
43085c3
use constants for target/pending status
drewmullen Jul 20, 2021
0ab1369
abstract nested error lookup
drewmullen Jul 20, 2021
b2f0a1d
create fsx gateway & tests
drewmullen Jun 4, 2021
7688f0d
add tag tests
drewmullen Jul 6, 2021
f886cfd
docs
drewmullen Jul 13, 2021
cd9434b
fix lint errors
drewmullen Jul 14, 2021
4f88a0d
SG-FSx include AuditArn if empty string.
AdamTylerLynch Jul 14, 2021
86b4a86
SG-FSx extract the lookup logic to the relevant finder package.
AdamTylerLynch Jul 14, 2021
b632f0d
SG-FSx added AccTest step to unset audit arn
AdamTylerLynch Jul 14, 2021
e0d8c0d
SG-FSx updated formatting in the docs
AdamTylerLynch Jul 14, 2021
02b07bc
SG-FSx added audit disabled AccTest and fixed formating issues.
AdamTylerLynch Jul 14, 2021
47423e4
SG-FSx modified AccTest for AuditArn
AdamTylerLynch Jul 14, 2021
6f95c01
SG-FSx updated the state change for update
AdamTylerLynch Jul 14, 2021
f195a21
GS-FSx modified waiter/stats to use new finder method
AdamTylerLynch Jul 15, 2021
88d3f14
add tests for dependencies disappearing
drewmullen Jul 16, 2021
5ffef2f
fixed doc errors
drewmullen Jul 19, 2021
d9d7746
use contants for fsx pending / target states
drewmullen Jul 20, 2021
5f0a239
add sgw arn to err, rm unnecessary setid()
drewmullen Jul 20, 2021
5157cd9
use constants for target/pending status
drewmullen Jul 20, 2021
1cc271c
abstract nested error lookup
drewmullen Jul 20, 2021
5d01764
rm unused variable, prefer inline
drewmullen Jul 20, 2021
2337c4e
positional params instead of repeating params
drewmullen Jul 20, 2021
0fe9ec3
rm unnecessary exported attrs
drewmullen Jul 20, 2021
1f7f890
use error pattern from errors.go
drewmullen Jul 20, 2021
76ee198
SG updating AccTests to use random DomainName as a fix for a merge re…
AdamTylerLynch Jul 20, 2021
27dea81
Merging in upstream changes from branch.
AdamTylerLynch Jul 20, 2021
c77975e
Merge in upstream changes
AdamTylerLynch Jul 20, 2021
3734b55
merging in upstream changes for SG SMB AccTests
AdamTylerLynch Jul 20, 2021
e49f62b
Fixed an issue where the hardcoded value of domain was still bring used.
AdamTylerLynch Jul 20, 2021
53daf47
updated AccTest composeConfig function calls
AdamTylerLynch Jul 21, 2021
ec476c2
finder use InvalidGatewayRequestErrCodeEquals
drewmullen Jul 21, 2021
6691fcb
use the same alias
drewmullen Jul 21, 2021
f30b4fe
use DefaultTimeout with params
drewmullen Jul 21, 2021
70d47da
use t.skip() instead of comment out test
drewmullen Jul 21, 2021
27b9cd3
rename resource
drewmullen Jul 21, 2021
57946f6
rename files and other fixes
drewmullen Jul 23, 2021
161228f
Tweaks function names and log messages
gdavison Jul 27, 2021
0a82556
Missed renaming file
gdavison Jul 27, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changelog/20082.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
```release-note:new-resource
resource/aws_storagegateway_fsx_associate_file_system: Add new resource for associating FSx Storage Gateway with FSx File Systems
```
```release-note:enhancement
resource/aws_storagegateway_gateway: Add new option for gateway_type, FILE_FSX_SMB, to be used with aws_storagegateway_fsx_associate_file_system
```
14 changes: 14 additions & 0 deletions aws/internal/service/storagegateway/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (

const (
OperationErrCodeFileShareNotFound = "FileShareNotFound"
FileSystemAssociationNotFound = "FileSystemAssociationNotFound"
)

// OperationErrorCode returns the operation error code from the specified error:
Expand All @@ -26,3 +27,16 @@ func OperationErrorCode(err error) string {

return ""
}

// Error code constants missing from AWS Go SDK:
// https://docs.aws.amazon.com/sdk-for-go/api/service/storagegateway/#pkg-constants

func InvalidGatewayRequestErrCodeEquals(err error, errCode string) bool {
var igrex *storagegateway.InvalidGatewayRequestException
if errors.As(err, &igrex) {
if err := igrex.Error_; err != nil {
return aws.StringValue(err.ErrorCode) == errCode
}
}
return false
}
40 changes: 39 additions & 1 deletion aws/internal/service/storagegateway/finder/finder.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
package finder

import (
"errors"
"fmt"
"log"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/storagegateway"
"github.com/hashicorp/aws-sdk-go-base/tfawserr"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
tfstoragegateway "github.com/terraform-providers/terraform-provider-aws/aws/internal/service/storagegateway"
)
Expand Down Expand Up @@ -109,6 +114,39 @@ func SMBFileShareByARN(conn *storagegateway.StorageGateway, arn string) (*storag

// TODO Check for multiple results.
// TODO https://github.com/hashicorp/terraform-provider-aws/pull/17613.

return output.SMBFileShareInfoList[0], nil
}

func FileSystemAssociationByARN(conn *storagegateway.StorageGateway, fileSystemAssociationARN string) (*storagegateway.FileSystemAssociationInfo, error) {

input := &storagegateway.DescribeFileSystemAssociationsInput{
FileSystemAssociationARNList: []*string{aws.String(fileSystemAssociationARN)},
}
log.Printf("[DEBUG] Reading Storage Gateway FSx File Associations: %s", input)

output, err := conn.DescribeFileSystemAssociations(input)
if err != nil {
if tfawserr.ErrCodeEquals(err, storagegateway.ErrCodeInvalidGatewayRequestException) {
var igrex *storagegateway.InvalidGatewayRequestException
if ok := errors.As(err, &igrex); ok {
if err := igrex.Error_; err != nil {
if aws.StringValue(err.ErrorCode) == "FileSystemAssociationNotFound" {
drewmullen marked this conversation as resolved.
Show resolved Hide resolved
drewmullen marked this conversation as resolved.
Show resolved Hide resolved
log.Printf("[WARN] FSX File System %q not found", fileSystemAssociationARN)
return nil, nil
}
}
}
}

return nil, fmt.Errorf("error reading Storage Gateway FSx File System: %w", err)
}

if output == nil || len(output.FileSystemAssociationInfoList) == 0 || output.FileSystemAssociationInfoList[0] == nil {
log.Printf("[WARN] FSX File System %q not found", fileSystemAssociationARN)
return nil, nil
}

filesystem := output.FileSystemAssociationInfoList[0]

return filesystem, nil
}
57 changes: 50 additions & 7 deletions aws/internal/service/storagegateway/waiter/status.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,21 @@ import (
)

const (
StorageGatewayGatewayStatusConnected = "GatewayConnected"
StoredIscsiVolumeStatusNotFound = "NotFound"
StoredIscsiVolumeStatusUnknown = "Unknown"
NfsFileShareStatusNotFound = "NotFound"
NfsFileShareStatusUnknown = "Unknown"
SmbFileShareStatusNotFound = "NotFound"
SmbFileShareStatusUnknown = "Unknown"
StorageGatewayGatewayStatusConnected = "GatewayConnected"
StoredIscsiVolumeStatusNotFound = "NotFound"
StoredIscsiVolumeStatusUnknown = "Unknown"
NfsFileShareStatusNotFound = "NotFound"
NfsFileShareStatusUnknown = "Unknown"
SmbFileShareStatusNotFound = "NotFound"
SmbFileShareStatusUnknown = "Unknown"
FsxFileSystemStatusNotFound = "NotFound"
FsxFileSystemStatusUnknown = "Unknown"
FsxFileSystemAssociationStatusAvailable = "AVAILABLE"
FsxFileSystemAssociationStatusCreating = "CREATING"
FsxFileSystemAssociationStatusDeleting = "DELETING"
FsxFileSystemAssociationStatusForceDeleting = "FORCE_DELETING"
FsxFileSystemAssociationStatusUpdating = "UPDATING"
FsxFileSystemAssociationStatusError = "ERROR"
)

func StorageGatewayGatewayStatus(conn *storagegateway.StorageGateway, gatewayARN string) resource.StateRefreshFunc {
Expand Down Expand Up @@ -128,3 +136,38 @@ func SMBFileShareStatus(conn *storagegateway.StorageGateway, arn string) resourc
return output, aws.StringValue(output.FileShareStatus), nil
}
}

func FsxFileSystemStatus(conn *storagegateway.StorageGateway, fileSystemArn string) resource.StateRefreshFunc {
return func() (interface{}, string, error) {

output, err := finder.FileSystemAssociationByARN(conn, fileSystemArn)

// there was an unhandled error in the Finder
if err != nil {
return nil, "", err
}

// no error, and no File System Association found
if output == nil {
return nil, FsxFileSystemStatusNotFound, nil
}

return output, aws.StringValue(output.FileSystemAssociationStatus), nil
}
}

func FsxFileSystemStatusAvailableStatusPending() []string {
return []string{FsxFileSystemAssociationStatusCreating, FsxFileSystemAssociationStatusUpdating}
}

func FsxFileSystemStatusAvailableStatusTarget() []string {
return []string{FsxFileSystemAssociationStatusAvailable}
}

func FsxFileSystemStatusDeletedStatusPending() []string {
return []string{FsxFileSystemAssociationStatusAvailable, FsxFileSystemAssociationStatusDeleting, FsxFileSystemAssociationStatusForceDeleting}
}

func FsxFileSystemStatusDeletedStatusTarget() []string {
return []string{}
}
40 changes: 40 additions & 0 deletions aws/internal/service/storagegateway/waiter/waiter.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ const (
NfsFileShareDeletedDelay = 5 * time.Second
SmbFileShareAvailableDelay = 5 * time.Second
SmbFileShareDeletedDelay = 5 * time.Second
FsxFileSystemAvailableDelay = 5 * time.Second
FsxFileSystemDeletedDelay = 5 * time.Second
)

func StorageGatewayGatewayConnected(conn *storagegateway.StorageGateway, gatewayARN string, timeout time.Duration) (*storagegateway.DescribeGatewayInformationOutput, error) {
Expand Down Expand Up @@ -166,3 +168,41 @@ func SMBFileShareUpdated(conn *storagegateway.StorageGateway, arn string, timeou

return nil, err
}

// FsxFileSystemAvailable waits for a FSx File System to return Available
func FsxFileSystemAvailable(conn *storagegateway.StorageGateway, fileSystemArn string, timeout time.Duration) (*storagegateway.FileSystemAssociationInfo, error) {
stateConf := &resource.StateChangeConf{
Pending: FsxFileSystemStatusAvailableStatusPending(),
Target: FsxFileSystemStatusAvailableStatusTarget(),
Refresh: FsxFileSystemStatus(conn, fileSystemArn),
Timeout: timeout,
Delay: FsxFileSystemAvailableDelay,
}

outputRaw, err := stateConf.WaitForState()

if output, ok := outputRaw.(*storagegateway.FileSystemAssociationInfo); ok {
return output, err
}

return nil, err
}

func FsxFileSystemDeleted(conn *storagegateway.StorageGateway, fileSystemArn string, timeout time.Duration) (*storagegateway.FileSystemAssociationInfo, error) {
stateConf := &resource.StateChangeConf{
Pending: FsxFileSystemStatusDeletedStatusPending(),
Target: FsxFileSystemStatusDeletedStatusTarget(),
Refresh: FsxFileSystemStatus(conn, fileSystemArn),
Timeout: timeout,
Delay: FsxFileSystemDeletedDelay,
NotFoundChecks: 1,
}

outputRaw, err := stateConf.WaitForState()

if output, ok := outputRaw.(*storagegateway.FileSystemAssociationInfo); ok {
return output, err
}

return nil, err
}
1 change: 1 addition & 0 deletions aws/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -1098,6 +1098,7 @@ func Provider() *schema.Provider {
"aws_ssoadmin_permission_set_inline_policy": resourceAwsSsoAdminPermissionSetInlinePolicy(),
"aws_storagegateway_cache": resourceAwsStorageGatewayCache(),
"aws_storagegateway_cached_iscsi_volume": resourceAwsStorageGatewayCachedIscsiVolume(),
"aws_storagegateway_fsx_associate_file_system": resourceAwsStorageGatewayFsxAssociateFileSystem(),
drewmullen marked this conversation as resolved.
Show resolved Hide resolved
"aws_storagegateway_gateway": resourceAwsStorageGatewayGateway(),
"aws_storagegateway_nfs_file_share": resourceAwsStorageGatewayNfsFileShare(),
"aws_storagegateway_smb_file_share": resourceAwsStorageGatewaySmbFileShare(),
Expand Down
2 changes: 1 addition & 1 deletion aws/resource_aws_storagegateway_cache_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ resource "aws_storagegateway_cache" "test" {
# We expect this data source value to change due to how Storage Gateway works.

lifecycle {
ignore_changes = ["disk_id"]
ignore_changes = [disk_id]
}

disk_id = data.aws_storagegateway_local_disk.test.id
Expand Down
Loading