Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable tags for Lambda functions #13352

Merged
merged 5 commits into from
Aug 27, 2019
Merged

Configurable tags for Lambda functions #13352

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
71a97eb
make tags configurable for aws lambda
kvch Aug 26, 2019
4ffba0f
add changelog entry
kvch Aug 26, 2019
e8e6e4e
Merge branch 'master' into feature-functionbeat-aws-tagging
kvch Aug 27, 2019
8912b7d
add validation for tags configuration
kvch Aug 27, 2019
0eb8533
adjust value limit
kvch Aug 27, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Export automation templates used to create functions. {pull}11923[11923]
- Configurable Amazon endpoint. {pull}12369[12369]
- Add timeout option to reference configuration. {pull}13351[13351]
- Configurable tags for Lambda functions. {pull}13352[13352]

*Winlogbeat*

Expand Down
12 changes: 12 additions & 0 deletions x-pack/functionbeat/_meta/beat.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ functionbeat.provider.aws.functions:
# Dead letter queue configuration, this must be set to an ARN pointing to a SQS queue.
#dead_letter_config.target_arn:

# Tags are key-value pairs attached to the function.
#tags:
# department: ops

# Optional fields that you can specify to add additional information to the
# output. Fields can be scalar values, arrays, dictionaries, or any nested
# combination of these.
Expand Down Expand Up @@ -98,6 +102,10 @@ functionbeat.provider.aws.functions:
# Dead letter queue configuration, this must be set to an ARN pointing to a SQS queue.
#dead_letter_config.target_arn:

# Tags are key-value pairs attached to the function.
#tags:
# department: ops

# Optional fields that you can specify to add additional information to the
# output. Fields can be scalar values, arrays, dictionaries, or any nested
# combination of these.
Expand Down Expand Up @@ -151,6 +159,10 @@ functionbeat.provider.aws.functions:
# Dead letter queue configuration, this must be set to an ARN pointing to a SQS queue.
#dead_letter_config.target_arn:

# Tags are key-value pairs attached to the function.
#tags:
# department: ops

# Optional fields that you can specify to add additional information to the
# output. Fields can be scalar values, arrays, dictionaries, or any nested
# combination of these.
Expand Down
12 changes: 12 additions & 0 deletions x-pack/functionbeat/functionbeat.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ functionbeat.provider.aws.functions:
# Dead letter queue configuration, this must be set to an ARN pointing to a SQS queue.
#dead_letter_config.target_arn:

# Tags are key-value pairs attached to the function.
#tags:
# department: ops

# Optional fields that you can specify to add additional information to the
# output. Fields can be scalar values, arrays, dictionaries, or any nested
# combination of these.
Expand Down Expand Up @@ -98,6 +102,10 @@ functionbeat.provider.aws.functions:
# Dead letter queue configuration, this must be set to an ARN pointing to a SQS queue.
#dead_letter_config.target_arn:

# Tags are key-value pairs attached to the function.
#tags:
# department: ops

# Optional fields that you can specify to add additional information to the
# output. Fields can be scalar values, arrays, dictionaries, or any nested
# combination of these.
Expand Down Expand Up @@ -151,6 +159,10 @@ functionbeat.provider.aws.functions:
# Dead letter queue configuration, this must be set to an ARN pointing to a SQS queue.
#dead_letter_config.target_arn:

# Tags are key-value pairs attached to the function.
#tags:
# department: ops

# Optional fields that you can specify to add additional information to the
# output. Fields can be scalar values, arrays, dictionaries, or any nested
# combination of these.
Expand Down
10 changes: 10 additions & 0 deletions x-pack/functionbeat/manager/aws/template_builder.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,15 @@ func (d *defaultTemplateBuilder) template(function installer, name, codeLoc stri
}
}

var tags []cloudformation.Tag
for name, val := range lambdaConfig.Tags {
tag := cloudformation.Tag{
Key: name,
Value: val,
}
tags = append(tags, tag)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add validation on the tags see.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-resource-tags.html

Looking at the documentation there are limit on lenght and they cannot start with aws:

Key

    The key name of the tag. You can specify a value that is 1 to 127 Unicode characters in length and cannot be prefixed with aws:. You can use any of the following characters: the set of Unicode letters, digits, whitespace, _, ., /, =, +, and -.

    Required: Yes

    Type: String
Value

    The value for the tag. You can specify a value that is 1 to 255 Unicode characters in length and cannot be prefixed with aws:. You can use any of the following characters: the set of Unicode letters, digits, whitespace, _, ., /, =, +, and -.

    Required: Yes

    Type: String

}

// Create the lambda
// Doc: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.html
template.Resources[prefix("")] = &AWSLambdaFunction{
Expand All @@ -190,6 +199,7 @@ func (d *defaultTemplateBuilder) template(function installer, name, codeLoc stri
MemorySize: lambdaConfig.MemorySize.Megabytes(),
ReservedConcurrentExecutions: lambdaConfig.Concurrency,
Timeout: int(lambdaConfig.Timeout.Seconds()),
Tags: tags,
},
DependsOn: dependsOn,
}
Expand Down
26 changes: 25 additions & 1 deletion x-pack/functionbeat/provider/aws/aws/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ package aws
import (
"fmt"
"regexp"
"strings"
"time"
"unicode"
"unicode/utf8"

humanize "github.com/dustin/go-humanize"

Expand Down Expand Up @@ -49,6 +51,7 @@ type LambdaConfig struct {
Timeout time.Duration `config:"timeout" validate:"nonzero,positive"`
Role string `config:"role"`
VPCConfig *vpcConfig `config:"virtual_private_cloud"`
Tags map[string]string `config:"tags"`
}

// Validate checks a LambdaConfig
Expand All @@ -65,7 +68,7 @@ func (c *LambdaConfig) Validate() error {
return fmt.Errorf("invalid role: '%s', name must match pattern %s", c.Role, arnRolePattern)
}

return nil
return validateTags(c.Tags)
}

type deadLetterConfig struct {
Expand All @@ -77,6 +80,27 @@ type vpcConfig struct {
SubnetIDs []string `config:"subnet_ids" validate:"required"`
}

func validateTags(tags map[string]string) error {
for key, val := range tags {
if strings.HasPrefix(key, "aws:") {
return fmt.Errorf("key '%s' cannot be prefixed with 'aws:'", key)
}
if strings.HasPrefix(val, "aws:") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "aws:" is only a limitation for keys not the values.

Limitation for the value is 255 not 127 (127 is only for the key) as mentioned in the comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"aws:" is limitation for both key and value. But you are right about the lenghts.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah crap you are right about the value, weird, things, I guess they scan the whole structure to make a decision.

return fmt.Errorf("value '%s' cannot be prefixed with 'aws:'", val)
}
keyLen := utf8.RuneCountInString(key)
if keyLen > 127 {
return fmt.Errorf("too long key; expected 127 chars, but got %d", keyLen)
}
valLen := utf8.RuneCountInString(val)
if valLen > 255 {
return fmt.Errorf("too long value; expected 255 chars, but got %d", valLen)
}
}

return nil
}

// MemSizeFactor64 implements a human understandable format for bytes but also make sure that all
// values used are a factory of 64.
type MemSizeFactor64 int
Expand Down