-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add worker lambda template that's deployed by AWS CDK #395
Merged
Merged
Changes from all commits
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
d1c21c7
add worker lambda template that's deployed by AWS CDK
damienung1 1b3150d
update base template test snapshot
damienung1 c37e1e1
Update template/lambda-sqs-worker-cdk/.me
damienung1 f473201
relax build container version to major version only
damienung1 afaeb62
follow style recommendation for runtype imports
damienung1 1a7f7e1
remove unused package
damienung1 8b1ac25
remove maxWorkers flag from test step
damienung1 d62dd06
add @seek/logger, log hello world only
damienung1 acb8a44
remove CDK generated object that's not actually being used
damienung1 9f34eeb
Update template/lambda-sqs-worker-cdk/.buildkite/pipeline.yml
damienung1 d8d3616
remove @types/jest as it's bundled in skuba
damienung1 d7286fe
remove code portion that's supposed to be populated by `skuba init`
damienung1 56dd771
update @types/node
damienung1 acac1f4
keep to default Dockerfile naming
damienung1 0b3ba19
Update README.md
damienung1 0a8a707
Update template/lambda-sqs-worker-cdk/infra/appStack.test.ts
damienung1 f0daac2
remove unnecessary shell out to bash
damienung1 a98046a
move CDK gitignores to base
damienung1 bf61a52
update snapshot test
damienung1 c788c18
Tweak base .gitignore
72636c 5658f51
Update template/lambda-sqs-worker-cdk/.nvmrc
damienung1 44bd211
remove bash and git install step from the build docker image
damienung1 211232e
upstream resolveJsonModule
damienung1 5bcd4ff
remove esbuild
damienung1 5b8bcf5
Update README.md
damienung1 5021fc8
Update template/lambda-sqs-worker-cdk/src/app.ts
damienung1 238e850
prune configuration that needs to be managed in CDK context
damienung1 b15ccc2
fix ECR caching and packaging
damienung1 bd4ae58
remove redundant shebang
damienung1 ef8f9d4
Merge branch 'master' into cdk-lambda-sqs-worker-template
72636c 357ac30
Create empty-shirts-film.md
72636c 4619139
Add to CI
72636c c93dca8
Add to init prompt
72636c File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
'skuba': minor | ||
--- | ||
|
||
**template/lambda-sqs-worker-cdk:** Add new template |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,8 @@ | ||
{ | ||
"compilerOptions": { | ||
"incremental": true, | ||
"moduleResolution": "node" | ||
"moduleResolution": "node", | ||
"resolveJsonModule": true | ||
}, | ||
"extends": "tsconfig-seek" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,9 @@ | ||
# managed by skuba | ||
.cdk.staging/ | ||
.idea/ | ||
.serverless/ | ||
.vscode/ | ||
cdk.out/ | ||
node_modules*/ | ||
|
||
/coverage*/ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
dev-agent: &dev-agent | ||
agents: | ||
queue: <%- devBuildkiteQueueName %> | ||
|
||
prod-agent: &prod-agent | ||
agents: | ||
queue: <%- prodBuildkiteQueueName %> | ||
|
||
plugins: &plugins #alias for shared plugins | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have some thoughts on this pipeline, but the |
||
seek-oss/aws-sm#v2.3.1: | ||
env: | ||
NPM_TOKEN: 'arn:aws:secretsmanager:ap-southeast-2:987872074697:secret:npm/npm-read-token' | ||
docker#v3.8.0: | ||
volumes: | ||
- /workdir/node_modules | ||
- /workdir/lib | ||
environment: | ||
- NPM_TOKEN | ||
seek-oss/docker-ecr-cache#v1.9.0: | ||
build-args: | ||
- NPM_TOKEN | ||
cache-on: | ||
- package.json | ||
- yarn.lock | ||
|
||
steps: | ||
- label: ':yarn: :eslint: Lint and :jest: unit test' | ||
<<: *dev-agent | ||
plugins: | ||
<<: *plugins | ||
key: test | ||
command: | ||
- echo "--- Running yarn lint" | ||
- yarn lint | ||
- echo "--- Running yarn test :jest:" | ||
- yarn test | ||
|
||
- label: 'CDK Deploy Staging :shipit:' | ||
<<: *dev-agent | ||
plugins: | ||
<<: *plugins | ||
depends_on: | ||
- test | ||
command: | ||
- echo "--- Running CDK deploy to staging" | ||
- yarn deploy:dev | ||
concurrency: 1 | ||
concurrency_group: '<%- repoName %>/deploy/dev' | ||
|
||
- label: 'CDK Deploy Production :shipit:' | ||
branches: 'master' | ||
<<: *prod-agent | ||
plugins: | ||
<<: *plugins | ||
depends_on: | ||
- test | ||
command: | ||
- echo "--- Running CDK deploy to production" | ||
- yarn deploy:prod | ||
concurrency: 1 | ||
concurrency_group: '<%- repoName %>/deploy/prod' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
--- | ||
# https://codex.ssod.skinfra.xyz/docs | ||
|
||
components: | ||
'<%- repoName %>': | ||
# TODO: supply system catalog information | ||
# dependencies: | ||
# - type: api | ||
# key: SEEK-Jobs/??? | ||
# - type: datastore | ||
# arn: arn:aws:dynamodb:us-east-1:123456789012:table/??? | ||
# - type: datastore | ||
# key: infrastructure/??? | ||
deploy_target: Worker | ||
# is_production_system: true | ||
primary_technologies: | ||
- AWS CDK | ||
- AWS Lambda | ||
- Buildkite | ||
- skuba | ||
- TypeScript | ||
# scope: APAC |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
14 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Docker image history includes ARG values, so never target this stage directly | ||
FROM node:14-alpine AS unsafe-dev-deps | ||
|
||
WORKDIR /workdir | ||
|
||
COPY package.json yarn.lock ./ | ||
|
||
ARG NPM_READ_TOKEN | ||
|
||
RUN \ | ||
echo '//registry.npmjs.org/:_authToken=${NPM_READ_TOKEN}' > .npmrc && \ | ||
yarn install --frozen-lockfile --ignore-optional --non-interactive && \ | ||
yarn package && \ | ||
rm .npmrc | ||
|
||
### | ||
|
||
FROM node:14-alpine AS dev-deps | ||
|
||
WORKDIR /workdir | ||
|
||
COPY --from=unsafe-dev-deps /workdir . | ||
|
||
### | ||
|
||
FROM dev-deps AS build | ||
|
||
COPY . . | ||
|
||
RUN yarn build |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
{ | ||
"app": "npx ts-node infra/index.ts", | ||
"context": { | ||
"@aws-cdk/core:enableStackNameDuplicates": "true", | ||
"global": { | ||
"appName": "<%- serviceName %>" | ||
}, | ||
"dev": { | ||
"workerLambda": { | ||
"reservedConcurrency": 1, | ||
"environment": { | ||
"SOMETHING": "dev" | ||
} | ||
} | ||
}, | ||
"prod": { | ||
"workerLambda": { | ||
"reservedConcurrency": 2, | ||
"environment": { | ||
"SOMETHING": "prod" | ||
} | ||
} | ||
} | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem in line with the Technology Strategy: https://tech-strategy.ssod.skinfra.xyz/docs/v1/technology.html#lambda
I don't mind merging this, but can we kick off internal discussions over the merit of the CDK as a Lambda deployment tool?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possibly a misunderstanding on my part:
https://tech-strategy.ssod.skinfra.xyz/docs/v1/components.html#infrastructure
I read it to include Lambda workers. We're already using CDK to deploy workers in practice (among other things).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think @72636c is referring to:
This also repeated under the Worker section:
And the Serverless section:
Tech Strategy allowing CDK for infrastructure wouldn’t override the above guidance.
If your team has decided to deviate from SEEK’s Tech Strategy you could BYO GitHub template instead of including it in Skuba’s default templates.