Populate Severity Fields from the AWS CloudWatch logs using transform processor

[ghost]

Component(s)

receiver/awscloudwatch

What happened?

Description

I am using the awscloudwatchreceiver to fetch logs from AWS CloudWatch.

A log entry looks something like this
[2023-09-22 07:38:22,570] INFO [GroupCoordinator 2]: Assignment received from leader consumer-amazon.msk.canary.group.broker-1-2686-5f4d4078-4257-41ed-a38d-baed4f17396f for group amazon.msk.canary.group.broker-1 for generation 100469. The group has 1 members, 0 of which are static. (kafka.coordinator.group.GroupCoordinator)

The awscloudwatchreceiver works fine and is able to pick logs.
But severity fields comes as
SeverityText: (empty)
SeverityNumber: Unspecified(0)

Expected Result

I want to pick the the log level and populate the severity fields from the log body itself, currently which looks like this.

Sep 22 07:06:32 ip-10-11-3-12.ap-south-1.compute.internal otelcol-contrib[13288]: Body: Str([2023-09-22 07:04:33,223] INFO [GroupCoordinator 1]: Assignment received from leader consumer-amazon.msk.canary.group.broker-3-2712-c7a8b2ba-2683-460e-be65-86f2091c147b for group amazon.msk.canary.group.broker-3 for generation 100413. The group has 1 members, 0 of which are static. (kafka.coordinator.group.GroupCoordinator))

Was hoping to achieve this using the transform processor, but not sure what exact query to be written. Can someone please help me with this ?

Thanks and Regards

Collector version

v0.83.0

Environment information

OS
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3⭕amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

OpenTelemetry Collector configuration

awscloudwatch:
  region: ap-south-1
  logs:
    poll_interval: 1m
    groups:
      named:
        /aws/kafka/flows-kafka-dev: 
          names:
            - flows-kafka-dev-772d11a9-4f67-4f3a-aa0b-f09dfdb09419-2-Broker-1
            - flows-kafka-dev-772d11a9-4f67-4f3a-aa0b-f09dfdb09419-2-Broker-2
            - flows-kafka-dev-772d11a9-4f67-4f3a-aa0b-f09dfdb09419-2-Broker-3

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/awscloudwatch: @djaglowski @schmikei

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

Pinging code owners for processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[TylerHelmuth]

@shiva-fyers I believe you can accomplish your goal using IsMatch. Checkout #27078

[ghost]

The Collector doesn't even start when using :
- set(severity_number, SEVERITY_NUMBER_DEBUG) where IsString(body) and IsMatch(body, "\sDEBUG\s")

but starts with :
- set(severity_number, SEVERITY_NUMBER_DEBUG) where IsString(body) and IsMatch(body, "DEBUG")

Is this related to this #23238

[TylerHelmuth]

Oh, I needed to do "\\sDEBUG\\s"

[TylerHelmuth]

#27098

[crobert-1]

@shiva-fyers Have you been able to try the IsMatch functionality again with the correct syntax?

[ghost]

Hey @crobert-1 I was able to run it. Forgot to close this. Apologies.

Thanks everyone.