Skip to content

Commit

Permalink
feat!: replace tslog by awspowertools logging (#3037)
Browse files Browse the repository at this point in the history
The module is using TSLog now for a sometime. But after breaking changes we are not able to update TSLog anymore. In the meantime, AWS have released [AWS Powertools for Lambda](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/core/logger/), including logging capabilities. Since this solution is AWS native, we are replacing the outdated TSLog by AWS Powertools logging.

## Breaking change
In this PR we replace TSLog by AWS Powertools logs with a minimal impact on code changes. The change can have be breaking in the following cases:
- Processing the logging, we only will support JSON formatted log
- Depending on the removed setting LOG_TYPE 
- Depending on the LOG_LEVEL silly, fatal or trace

## Log messages
All log messages are JSON formatted and contain besides the message scope information like: `function-name`, `environment`, `module`. By enabling DEBUG log level the event received by the Lambda will be logged.
  • Loading branch information
npalm authored Mar 22, 2023
1 parent 9909d90 commit 4c3ee20
Show file tree
Hide file tree
Showing 60 changed files with 1,551 additions and 1,740 deletions.
52 changes: 50 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

This [Terraform](https://www.terraform.io/) module creates the required infrastructure needed to host [GitHub Actions](https://github.com/features/actions) self-hosted, auto-scaling runners on [AWS spot instances](https://aws.amazon.com/ec2/spot/). It provides the required logic to handle the life cycle for scaling up and down using a set of AWS Lambda functions. Runners are scaled down to zero to avoid costs when no workflows are active.

> 📢 [`v3`](https://github.com/philips-labs/terraform-aws-github-runner/pull/3037) underlying loggin framework is replaces by [AWS Lambda Powertools](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/). Depending on how you handle the logging of the module a migration could be required.
> 📢 [`v2`](https://github.com/philips-labs/terraform-aws-github-runner/issues/2517) support via a Terraform [submodule](./modules/multi-runner/README.md) an option to create multiple runners at once. The webhook will deliver based on matching rules the events to a dedicated queue for the runners. Next per queue a scaling lambda will ensure the specific runner is created. For more details checkout the [examples](./examples/multi-runner/README.md).
> 📢 `v1` is available on a dedicated branch. The default branch `main` is related to `v2`, for fixes or backports you can submit a PR to the branch `v1`. For feature PR's we will ask you to at least submit a PR to `main`
Expand Down Expand Up @@ -31,6 +33,7 @@ This [Terraform](https://www.terraform.io/) module creates the required infrastr
- [Experimental - Optional queue to publish GitHub workflow job events](#experimental---optional-queue-to-publish-github-workflow-job-events)
- [Examples](#examples)
- [Sub modules](#sub-modules)
- [Logging](#logging)
- [Debugging](#debugging)
- [Security Consideration](#security-consideration)
- [Requirements](#requirements)
Expand All @@ -41,6 +44,12 @@ This [Terraform](https://www.terraform.io/) module creates the required infrastr
- [Outputs](#outputs)
- [Contribution](#contribution)
- [Philips Forest](#philips-forest)
- [Requirements](#requirements-1)
- [Providers](#providers-1)
- [Modules](#modules-1)
- [Resources](#resources-1)
- [Inputs](#inputs-1)
- [Outputs](#outputs-1)

## Motivation

Expand Down Expand Up @@ -384,6 +393,45 @@ The following sub modules are optional and are provided as example or utility:

ARM64 configuration for submodules. When using the top level module configure `runner_architecture = "arm64"` and ensure the list of `instance_types` matches. When not using the top-level, ensure these properties are set on the submodules.

## Logging

The module uses [AWS Lambda Powertools](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/) for logging. By default the log level is set to `info`, by setting the leg level to `debug` the incoming event of the Lambda is logged as well.

Log messages contains at least the following keys:

- `messages`: The logged messages
- `environment`: The environment prefix provided via Terraform
- `service`: The lambda
- `module`: The TypeScript module writing the log message
- `function-name`: The name of the lambda function (prefix + function name)
- `github`: Depending on the lambda, contains GitHub context
- `runner`: Depending on the lambda, specific context related to the runner

An example log messages of the scale-up function:

```json
{
"level": "INFO",
"message": "Received event",
"service": "runners-scale-up",
"timestamp": "2023-03-20T08:15:27.448Z",
"xray_trace_id": "1-6418161e-08825c2f575213ef760531bf",
"module": "scale-up",
"region": "eu-west-1",
"environment": "my-linux-x64",
"aws-request-id": "eef1efb7-4c07-555f-9a67-b3255448ee60",
"function-name": "my-linux-x64-scale-up",
"runner": {
"type": "Repo",
"owner": "test-runners/multi-runner"
},
"github": {
"event": "workflow_job",
"workflow_job_id": "1234"
}
}
```

## Debugging

In case the setup does not work as intended follow the trace of events:
Expand Down Expand Up @@ -492,7 +540,7 @@ We welcome any improvement to the standard module to make the default as secure
| <a name="input_lambda_security_group_ids"></a> [lambda\_security\_group\_ids](#input\_lambda\_security\_group\_ids) | List of security group IDs associated with the Lambda function. | `list(string)` | `[]` | no |
| <a name="input_lambda_subnet_ids"></a> [lambda\_subnet\_ids](#input\_lambda\_subnet\_ids) | List of subnets in which the action runners will be launched, the subnets needs to be subnets in the `vpc_id`. | `list(string)` | `[]` | no |
| <a name="input_log_level"></a> [log\_level](#input\_log\_level) | Logging level for lambda logging. Valid values are 'silly', 'trace', 'debug', 'info', 'warn', 'error', 'fatal'. | `string` | `"info"` | no |
| <a name="input_log_type"></a> [log\_type](#input\_log\_type) | Logging format for lambda logging. Valid values are 'json', 'pretty', 'hidden'. | `string` | `"pretty"` | no |
| <a name="input_log_type"></a> [log\_type](#input\_log\_type) | Logging format for lambda logging. Valid values are 'json', 'pretty', 'hidden'. | `string` | `null` | no |
| <a name="input_logging_kms_key_id"></a> [logging\_kms\_key\_id](#input\_logging\_kms\_key\_id) | Specifies the kms key id to encrypt the logs with | `string` | `null` | no |
| <a name="input_logging_retention_in_days"></a> [logging\_retention\_in\_days](#input\_logging\_retention\_in\_days) | Specifies the number of days you want to retain log events for the lambda log group. Possible values are: 0, 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. | `number` | `180` | no |
| <a name="input_minimum_running_time_in_minutes"></a> [minimum\_running\_time\_in\_minutes](#input\_minimum\_running\_time\_in\_minutes) | The time an ec2 action runner should be running at minimum before terminated if not busy. | `number` | `null` | no |
Expand Down Expand Up @@ -742,4 +790,4 @@ Talk to the forestkeepers in the `runners`-channel on Slack.
| <a name="output_runners"></a> [runners](#output\_runners) | n/a |
| <a name="output_ssm_parameters"></a> [ssm\_parameters](#output\_ssm\_parameters) | n/a |
| <a name="output_webhook"></a> [webhook](#output\_webhook) | n/a |
<!-- END_TF_DOCS -->
<!-- END_TF_DOCS -->
3 changes: 3 additions & 0 deletions examples/default/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -90,4 +90,7 @@ module "runners" {

# prefix GitHub runners with the environment name
runner_name_prefix = "${local.environment}_"

# Enable debug logging for the lambda functions
# log_level = "debug"
}
4 changes: 2 additions & 2 deletions examples/ephemeral/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ module "runners" {
# data "aws_caller_identity" "current" {}
# ami_owners = [data.aws_caller_identity.current.account_id]

# Enable logging
log_level = "debug"
# Enable debug logging for the lambda functions
# log_level = "debug"

# Setup a dead letter queue, by default scale up lambda will kepp retrying to process event in case of scaling error.
# redrive_policy_build_queue = {
Expand Down
1 change: 1 addition & 0 deletions examples/multi-runner/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ module "multi-runner" {
# enable_workflow_job_events_queue = true
# override delay of events in seconds

# Enable debug logging for the lambda functions
# log_level = "debug"

}
3 changes: 0 additions & 3 deletions main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,6 @@ module "webhook" {
lambda_security_group_ids = var.lambda_security_group_ids
aws_partition = var.aws_partition

log_type = var.log_type
log_level = var.log_level
}

Expand Down Expand Up @@ -267,7 +266,6 @@ module "runners" {

kms_key_arn = var.kms_key_arn

log_type = var.log_type
log_level = var.log_level

pool_config = var.pool_config
Expand Down Expand Up @@ -308,7 +306,6 @@ module "runner_binaries" {
role_path = var.role_path
role_permissions_boundary = var.role_permissions_boundary

log_type = var.log_type
log_level = var.log_level

lambda_subnet_ids = var.lambda_subnet_ids
Expand Down
Loading

0 comments on commit 4c3ee20

Please sign in to comment.