Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assets: upload to and grant permissions on prefix #510

Merged
merged 8 commits into from
Aug 20, 2018
Merged

Assets: upload to and grant permissions on prefix #510

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a0ed0f8
Assets: upload to and grant permissions on prefix
Aug 3, 2018
6ce1fa0
Merge remote-tracking branch 'origin/master' into huijbers/asset-prefix
Aug 16, 2018
b5c3f6f
Toolkit: add caching for AWS credential providers.
Aug 16, 2018
3a7dac9
Combine key and prefix back into a single parameter
Aug 16, 2018
7eb3d71
Fix Lambda asset test
Aug 16, 2018
dacefbb
WIP
Aug 17, 2018
24a445c
Add unit test on assets
Aug 17, 2018
f88dcb3
Update integ expectations
Aug 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions packages/@aws-cdk/assets/lib/asset.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ export class Asset extends cdk.Construct {

private readonly bucket: s3.BucketRef;

private readonly s3PrefixParam: cdk.Parameter;

constructor(parent: cdk.Construct, id: string, props: GenericAssetProps) {
super(parent, id);

Expand All @@ -84,33 +86,45 @@ export class Asset extends cdk.Construct {
description: `S3 bucket for asset "${this.path}"`,
});

const keyParam = new cdk.Parameter(this, 'S3ObjectKey', {
this.s3PrefixParam = new cdk.Parameter(this, 'S3Prefix', {
type: 'String',
description: `S3 prefix for asset "${this.path}"`
});

const keyParam = new cdk.Parameter(this, 'S3VersionKey', {
type: 'String',
description: `S3 object for asset "${this.path}"`
description: `S3 key for asset version "${this.path}"`
});

this.s3BucketName = bucketParam.value;
this.s3ObjectKey = keyParam.value;

// grant the lambda's role read permissions on the code s3 object

this.bucket = s3.BucketRef.import(parent, 'AssetBucket', {
bucketName: this.s3BucketName
});

// form the s3 URL of the object key
this.s3Url = this.bucket.urlForObject(this.s3ObjectKey);

// Get a unique identifier for this asset, which will be used
// as a folder to group different versions of the same asset together.
//
// Even though this code looks horrible, this is actually not a terrible thing
// to do. The generated ID will contain the *stack name* as well, which is a
// perfectly nice thing to do to disambiguate similarly named assets in different stacks.
const uniqueId = new cdk.HashedAddressingScheme().allocateAddress(this.path.split('/'));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I keep wondering if we should just expose a uniqueId in the Construct level (and use for resource.logicalId). This keeps coming back and I think might be useful generally, and not only for CloudFormation resource identifiers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but their scopes are different, which complicates it a bit.

In a tree like this:

root
└── Stack
      └── Construct
             └── Construct

    ├─────── path ───────┤

         ├─ logical id ──┤

The .path is from the root to the construct, but the logical ID is only calculated off the nearest enclosing stack to the construct. If we didn't do that, all enclosing logical IDs would being with the stack name, which is not terrible for the IDs that they are, but they're also used to generate human-readable names, where that would be quite ugly (and in fact CFN sometimes--but not always!--prepends the stack name itself as well).

So we should probably still keep them separated, or have something like uniqueIdRelativeTo(ancestor)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could allow constructs to opt-out of the human portion of name generation, so they will be "skipped" when producing the human portion (something like hideName = true).


// attach metadata to the lambda function which includes information
// for tooling to be able to package and upload a directory to the
// s3 bucket and plug in the bucket name and key in the correct
// parameters.

const asset: cxapi.AssetMetadataEntry = {
path: this.assetPath,
id: uniqueId,
packaging: props.packaging,
s3BucketParameter: bucketParam.logicalId,
s3KeyParameter: keyParam.logicalId,
s3PrefixParameter: this.s3PrefixParam.logicalId
};

this.addMetadata(cxapi.ASSET_METADATA, asset);
Expand All @@ -124,7 +138,11 @@ export class Asset extends cdk.Construct {
* Grants read permissions to the principal on the asset's S3 object.
*/
public grantRead(principal?: iam.IPrincipal) {
this.bucket.grantRead(principal, this.s3ObjectKey);
// We give permissions on all files with the same prefix. Presumably
// different versions of the same file will have the same prefix
// and we don't want to accidentally revoke permission on old versions
// when deploying a new version.
this.bucket.grantRead(principal, new cdk.FnConcat(this.s3PrefixParam.value, "*"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should implement #24

}
}

Expand Down
18 changes: 15 additions & 3 deletions packages/@aws-cdk/assets/test/integ.assets.directory.lit.expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@
"Type": "String",
"Description": "S3 bucket for asset \"aws-cdk-asset-test/SampleAsset\""
},
"SampleAssetS3ObjectKey6F5D200B": {
"SampleAssetS3Prefix6E89595F": {
"Type": "String",
"Description": "S3 object for asset \"aws-cdk-asset-test/SampleAsset\""
"Description": "S3 prefix for asset \"aws-cdk-asset-test/SampleAsset\""
},
"SampleAssetS3VersionKey3E106D34": {
"Type": "String",
"Description": "S3 key for asset version \"aws-cdk-asset-test/SampleAsset\""
}
},
"Resources": {
Expand Down Expand Up @@ -76,7 +80,15 @@
},
"/",
{
"Ref": "SampleAssetS3ObjectKey6F5D200B"
"Fn::Join": [
"",
[
{
"Ref": "SampleAssetS3Prefix6E89595F"
},
"*"
]
]
}
]
]
Expand Down
18 changes: 15 additions & 3 deletions packages/@aws-cdk/assets/test/integ.assets.file.lit.expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@
"Type": "String",
"Description": "S3 bucket for asset \"aws-cdk-asset-file-test/SampleAsset\""
},
"SampleAssetS3ObjectKey6F5D200B": {
"SampleAssetS3Prefix6E89595F": {
"Type": "String",
"Description": "S3 object for asset \"aws-cdk-asset-file-test/SampleAsset\""
"Description": "S3 prefix for asset \"aws-cdk-asset-file-test/SampleAsset\""
},
"SampleAssetS3VersionKey3E106D34": {
"Type": "String",
"Description": "S3 key for asset version \"aws-cdk-asset-file-test/SampleAsset\""
}
},
"Resources": {
Expand Down Expand Up @@ -76,7 +80,15 @@
},
"/",
{
"Ref": "SampleAssetS3ObjectKey6F5D200B"
"Fn::Join": [
"",
[
{
"Ref": "SampleAssetS3Prefix6E89595F"
},
"*"
]
]
}
]
]
Expand Down
18 changes: 15 additions & 3 deletions packages/@aws-cdk/assets/test/integ.assets.permissions.lit.expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@
"Type": "String",
"Description": "S3 bucket for asset \"aws-cdk-asset-refs/MyFile\""
},
"MyFileS3ObjectKey4641930D": {
"MyFileS3PrefixF078A585": {
"Type": "String",
"Description": "S3 object for asset \"aws-cdk-asset-refs/MyFile\""
"Description": "S3 prefix for asset \"aws-cdk-asset-refs/MyFile\""
},
"MyFileS3VersionKey568C3C9F": {
"Type": "String",
"Description": "S3 key for asset version \"aws-cdk-asset-refs/MyFile\""
}
},
"Resources": {
Expand Down Expand Up @@ -76,7 +80,15 @@
},
"/",
{
"Ref": "MyFileS3ObjectKey4641930D"
"Fn::Join": [
"",
[
{
"Ref": "MyFileS3PrefixF078A585"
},
"*"
]
]
}
]
]
Expand Down
22 changes: 17 additions & 5 deletions packages/@aws-cdk/assets/test/integ.assets.refs.lit.expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@
"Type": "String",
"Description": "S3 bucket for asset \"aws-cdk-asset-refs/SampleAsset\""
},
"SampleAssetS3ObjectKey6F5D200B": {
"SampleAssetS3Prefix6E89595F": {
"Type": "String",
"Description": "S3 object for asset \"aws-cdk-asset-refs/SampleAsset\""
"Description": "S3 prefix for asset \"aws-cdk-asset-refs/SampleAsset\""
},
"SampleAssetS3VersionKey3E106D34": {
"Type": "String",
"Description": "S3 key for asset version \"aws-cdk-asset-refs/SampleAsset\""
}
},
"Outputs": {
Expand All @@ -20,7 +24,7 @@
},
"S3ObjectKey": {
"Value": {
"Ref": "SampleAssetS3ObjectKey6F5D200B"
"Ref": "SampleAssetS3VersionKey3E106D34"
},
"Export": {
"Name": "aws-cdk-asset-refs:S3ObjectKey"
Expand All @@ -46,7 +50,7 @@
},
"/",
{
"Ref": "SampleAssetS3ObjectKey6F5D200B"
"Ref": "SampleAssetS3VersionKey3E106D34"
}
]
]
Expand Down Expand Up @@ -123,7 +127,15 @@
},
"/",
{
"Ref": "SampleAssetS3ObjectKey6F5D200B"
"Fn::Join": [
"",
[
{
"Ref": "SampleAssetS3Prefix6E89595F"
},
"*"
]
]
}
]
]
Expand Down
Loading