Removed memory_ballast and making sure soft memory is set

[samiura]

Description: Removed memory_ballast and making sure soft memory is set

Link to Splunk idea: <Link to Splunk idea, see https://ideas.splunk.com>

Testing:

Documentation:

[dmitryax]

Do you want to do removal of the ballast extension from the configs in this PR or separately?

[samiura]

@dmitryax Removed ballast_extension from configs.

[jinja2]

Please add a changelog entry.

We are planning to update the default configs here in a separate PR? We'll also need to address ansible, chef, etc. deployments after this is available. They seem to set env var for ballast size.

[dmitryax]

As @jinja2 mentioned, we need to remove the memory_balast from all the configs like https://github.com/signalfx/splunk-otel-collector/blob/main/cmd/otelcol/config/collector/agent_config.yaml. It should be done along with this change, probably the same PR. Please make sure SPLUNK_BALLAST_SIZE_MIB env var isn't referenced anymore across the repo.

[samiura]

@jinja2 and @dmitryax , I will remove all the memory_ballast from configs. i was not sure if that was part of the scope of this PR.

[dmitryax]

Please rebase

[dmitryax]

@samiura are the failing integration tests related to the change?

[samiura]

@dmitryax No, if you look at the history, they have been failing ever since start of the last week. I have manually deployed this branch on my machine, kind cluster and observed the memory/cpu usage, I do not see any issues thus far.

[samiura]

@dmitryax As we speak this is the state of integration test in the main branch.

https://github.com/signalfx/splunk-otel-collector/actions/runs/8425269973/job/23071206244?pr=4413#step:7:7

[dmitryax]

LGTM. @samiura please proceed with the follow-up cleanup as we agreed