Add filebeat receiver and otel consumer output

[leehinman]

Proposed commit message

This PR adds a filebeat receiver and an otelconsumer output. These can be used together to run filebeat as an otel receiver.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

None

Author's Checklist

• [ ]

How to test this PR locally

receiver_test.go shows how receiver is created, started and stopped.

factories.Receivers, err = receiver.MakeFactoryMap(
                fbreceiver.NewFactory(),
        )

could be used in the opentelemetry collector to add the filebeat receiver.

Configuration would look like:

receivers:
  filebeatreceiver:
    filebeat:
      inputs:
        - type: benchmark
          enabled: true
          message: "test"
		  eps: 5
    output:
      otelconsumer:
    logging:
      level: debug
      selectors:
        - '*'
    processors:
      - add_fields:
          target: project
          fields:
            name: myproject
processors:
  batch:
exporters:
  debug:
    verbosity: detailed
    sampling_initial: 100
service:
  pipelines:
    logs:
      receivers: [filebeatreceiver]
      processors: [batch]
      exporters: [debug]

Related issues

Use cases

Screenshots

Logs

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[blakerouse]

I think adding more to the validation for the configuration would be nice to have. Like validating that the only output configured is the otelconsumer. Possibly injecting an the otelconsumer in the case that it was not defined in the configuration.

But I don't want to block on that here, as I think we can keep iterating. Lets get this merged so we can start testing.

[cmacknz]

If this is blocking further progress I am OK approving this without additional tests, but let's track the need for more testing in a GH issue first so we don't just forget to do it.

[blakerouse]

@cmacknz Definitely want to get this merged now, so we can get this being used by elastic-agent. I agree that more unit testing in that area would be good, but shouldn't block this PR. I filed this issue to track the additional coverage - #40977

Integration testing in elastic-agent will also be added for more coverage.

[blakerouse]

@cmacknz Looks like this needs a @elastic/beats-tech-leads review to get this merged.

[cmacknz]

Approved to unblock

[mauri870]

Codewise looks good, eager to have all the pieces together!