feat(glue-alpha): adding partition projection

[guyernest]

The partition projection feature in Athena and Glue needed to be supported correctly in the S3Table implementation. The table properties from the partition project were not added to the Parameters section of TableInput.

The new implementation allows the definition of the different types of partition projection, including date, integer and enum,

Closes: #27365

[guyernest]

This is the 3rd try of PR after:
#27447 Closed
#27485 Closed

[aws-cdk-automation]

This PR has been in the BUILD FAILING state for 3 weeks, and looks abandoned. To keep this PR from being closed, please continue work on it. If not, it will automatically be closed in a week.

[aws-cdk-automation]

This PR has been in the BUILD FAILING state for 3 weeks, and looks abandoned. To keep this PR from being closed, please continue work on it. If not, it will automatically be closed in a week.

[guyernest]

Is there someone from the maintainers/reviewers who can help with pushing this forward? I don't know how to get it to pass the build, as I don't have access to this environment and I don't see why it is failing.

[vinayak-kukreja]

Hey, apologies for the delay. Someone from the team will take a look at this.

At a brief glance, I see in the logs,

@aws-cdk/aws-glue-alpha: /codebuild/output/src1182002498/src/github.com/aws/aws-cdk/packages/@aws-cdk/aws-glue-alpha/lib/partition-projection.ts
@aws-cdk/aws-glue-alpha:    12:37   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    13:2    error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    19:51   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    23:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    24:1    error  Expected indentation of 4 spaces but found 6                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    25:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    26:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    27:1    error  Expected indentation of 4 spaces but found 6                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    28:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    28:6    error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    32:125  error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    33:72   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    51:35   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    52:36   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    55:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    56:1    error  Expected indentation of 4 spaces but found 6                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    57:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    57:43   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    58:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    58:20   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    59:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    60:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    64:1    error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    66:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    67:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    68:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    69:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    70:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    71:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    72:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    79:40   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    81:35   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    83:37   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    84:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    84:5    error  Opening curly brace does not appear on the same line as controlling statement  brace-style
@aws-cdk/aws-glue-alpha:    85:1    error  Expected indentation of 4 spaces but found 6                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    86:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    86:46   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    87:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    88:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    89:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    91:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    92:1    error  Expected indentation of 4 spaces but found 6                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    93:1    error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:    94:1    error  Expected indentation of 4 spaces but found 6                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    95:1    error  Expected indentation of 6 spaces but found 10                                  @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    96:1    error  Expected indentation of 6 spaces but found 10                                  @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    97:1    error  Expected indentation of 6 spaces but found 10                                  @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    98:1    error  Expected indentation of 6 spaces but found 10                                  @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:    99:1    error  Expected indentation of 6 spaces but found 10                                  @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   100:1    error  Expected indentation of 6 spaces but found 10                                  @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   101:1    error  Expected indentation of 4 spaces but found 6                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   102:1    error  Expected indentation of 2 spaces but found 4                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   106:1    error  More than 1 blank line not allowed                                             no-multiple-empty-lines
@aws-cdk/aws-glue-alpha:   109:40   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:   113:41   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:   115:30   error  Missing trailing comma                                                         comma-dangle
@aws-cdk/aws-glue-alpha:   121:1    error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:   123:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   124:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   125:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   126:1    error  Expected indentation of 6 spaces but found 8                                   @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   134:40   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha:   135:52   error  Unexpected trailing comma                                                      comma-dangle
@aws-cdk/aws-glue-alpha:   137:45   error  Trailing spaces not allowed                                                    no-trailing-spaces
@aws-cdk/aws-glue-alpha: /codebuild/output/src1182002498/src/github.com/aws/aws-cdk/packages/@aws-cdk/aws-glue-alpha/test/integ.table.ts
@aws-cdk/aws-glue-alpha:   140:4  error  Missing trailing comma  comma-dangle
@aws-cdk/aws-glue-alpha: /codebuild/output/src1182002498/src/github.com/aws/aws-cdk/packages/@aws-cdk/aws-glue-alpha/test/partition-projection.test.ts
@aws-cdk/aws-glue-alpha:    1:9   error  Trailing spaces not allowed                      no-trailing-spaces
@aws-cdk/aws-glue-alpha:    2:27  error  Trailing spaces not allowed                      no-trailing-spaces
@aws-cdk/aws-glue-alpha:    3:27  error  Trailing spaces not allowed                      no-trailing-spaces
@aws-cdk/aws-glue-alpha:    4:30  error  Expected a line break before this closing brace  object-curly-newline
@aws-cdk/aws-glue-alpha:    6:1   error  More than 1 blank line not allowed               no-multiple-empty-lines
@aws-cdk/aws-glue-alpha:   11:14  error  Trailing spaces not allowed                      no-trailing-spaces
@aws-cdk/aws-glue-alpha:   26:14  error  Trailing spaces not allowed                      no-trailing-spaces
@aws-cdk/aws-glue-alpha:   28:14  error  Trailing spaces not allowed                      no-trailing-spaces
@aws-cdk/aws-glue-alpha:   34:35  error  Strings must use singlequote                     quotes
@aws-cdk/aws-glue-alpha:   41:14  error  Trailing spaces not allowed                      no-trailing-spaces
@aws-cdk/aws-glue-alpha:   52:1   error  More than 1 blank line not allowed               no-multiple-empty-lines
@aws-cdk/aws-glue-alpha: /codebuild/output/src1182002498/src/github.com/aws/aws-cdk/packages/@aws-cdk/aws-glue-alpha/test/s3-table.test.ts
@aws-cdk/aws-glue-alpha:   1104:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1105:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1106:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1107:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1108:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1109:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1111:1   error  Trailing spaces not allowed                                no-trailing-spaces
@aws-cdk/aws-glue-alpha:   1121:1   error  Trailing spaces not allowed                                no-trailing-spaces
@aws-cdk/aws-glue-alpha:   1142:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1143:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1144:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1145:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1146:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1147:1   error  Expected indentation of 6 spaces but found 8               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1149:1   error  Trailing spaces not allowed                                no-trailing-spaces
@aws-cdk/aws-glue-alpha:   1151:37  error  Trailing spaces not allowed                                no-trailing-spaces
@aws-cdk/aws-glue-alpha:   1173:1   error  Trailing spaces not allowed                                no-trailing-spaces
@aws-cdk/aws-glue-alpha:   1180:9   error  Trailing spaces not allowed                                no-trailing-spaces
@aws-cdk/aws-glue-alpha:   1183:51  error  Trailing spaces not allowed                                no-trailing-spaces
@aws-cdk/aws-glue-alpha:   1185:11  error  Inconsistently quoted property 'classification' found      quote-props
@aws-cdk/aws-glue-alpha:   1186:11  error  Inconsistently quoted property 'has_encrypted_data' found  quote-props
@aws-cdk/aws-glue-alpha:   1186:36  error  Trailing spaces not allowed                                no-trailing-spaces
@aws-cdk/aws-glue-alpha:   1197:1   error  Expected indentation of 12 spaces but found 11             @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1197:18  error  Strings must use singlequote                               quotes
@aws-cdk/aws-glue-alpha:   1198:1   error  Expected indentation of 12 spaces but found 11             @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1198:18  error  Strings must use singlequote                               quotes
@aws-cdk/aws-glue-alpha:   1198:26  error  Missing trailing comma                                     comma-dangle
@aws-cdk/aws-glue-alpha:   1199:12  error  Missing trailing comma                                     comma-dangle
@aws-cdk/aws-glue-alpha:   1200:1   error  Expected indentation of 8 spaces but found 9               @typescript-eslint/indent
@aws-cdk/aws-glue-alpha:   1224:31  error  Strings must use singlequote                               quotes
@aws-cdk/aws-glue-alpha:   1224:37  error  Missing trailing comma                                     comma-dangle
@aws-cdk/aws-glue-alpha: ✖ 108 problems (108 errors, 0 warnings)
@aws-cdk/aws-glue-alpha:   108 errors and 0 warnings potentially fixable with the `--fix` option.
@aws-cdk/aws-glue-alpha: Error: /codebuild/output/src1182002498/src/github.com/aws/aws-cdk/node_modules/eslint/bin/eslint.js . --ext=.ts --resolve-plugins-relative-to=/codebuild/output/src1182002498/src/github.com/aws/aws-cdk/tools/@aws-cdk/cdk-build-tools/lib exited with error code 1
@aws-cdk/aws-glue-alpha: Build failed.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Looks like these are linting errors. I think these could be fixed with the lint command: https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-glue-alpha/package.json#L53.

[vinayak-kukreja]

I don't have access to this environment and I don't see why it is failing

Hey, were you not able to access the build logs? We posted a link in this comment: #27490 (comment)

[guyernest]

Thank you for the prompt reply over Slack. I fixed the linting issues. I hope that the build will run now and pass successfully.

[guyernest]

The build failed again, and I could access the build log as suggested. It wasn't helpful as the logs are long and full of errors that I have no control over. I didn't find any linting errors as were detected previously by @vinayak-kukreja . Any additional suggestions are welcome.

[vinayak-kukreja]

Hey, @guyernest I see the following error,

@aws-cdk/aws-glue-alpha: README.md:293 MD028/no-blanks-blockquote Blank line inside blockquote
@aws-cdk/aws-glue-alpha: Error: ''"'"'/usr/bin/node'"'"'' -- /codebuild/output/src2608913370/src/github.com/aws/aws-cdk/node_modules/markdownlint-cli/markdownlint.js --config /codebuild/output/src2608913370/src/github.com/aws/aws-cdk/tools/@aws-cdk/cdk-build-tools/config/markdownlint.json README.md exited with error code 1
@aws-cdk/aws-glue-alpha: Build failed.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
@aws-cdk/aws-glue-alpha: �[2K�[1G@aws-cdk/aws-glue-alpha: �[31merror�[39m Command failed with exit code 1.
@aws-cdk/aws-glue-alpha: �[2K�[1G@aws-cdk/aws-glue-alpha: �[34minfo�[39m Visit �[1mhttps://yarnpkg.com/en/docs/cli/run�[22m for documentation about this command.

I am curious, are you seeing these fail in PR builds or local builds too?

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: e63eaa6
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[guyernest]

I give up. I don't have time to waste on adding docstring to YEARS as it is a public enum and other housekeeping. If you find ways to automate some of this stuff, I'll be happy to contribute. Until then, good luck.

[guyernest]

I give up. I don't have time to waste on adding docstring to YEARS as it is a public enum and other housekeeping. If you find ways to automate some of this stuff, I'll be happy to contribute. Until then, good luck.