Percentage settings for memory limiter processor and ballast size

[pavolloffay]

Is your feature request related to a problem? Please describe.

Flag --mem-ballast-size-mib and memory linter processor configuration accept absolute settings. This makes it hard to enable and configure these components in a default/hardcoded configuration. The percentage settings could be applied to a wide range of deployments and default values could be used.

--mem-ballast-size-mib=2000

processors:
  memory_limiter:
    ballast_size_mib: 2000
    check_interval: 5s
    limit_mib: 4000
    spike_limit_mib: 500

Describe the solution you'd like

Use percentages to set --mem-ballast-size-mib, ballast_size_mib, limit_mib and spike_limit_mib. The check_interval could be set some optimal default value.

Describe alternatives you've considered

Wrap the memory linter and dynamically calculate absolute values. However, it would be better to get this implemented directly in the core.

Additional context

Jaeger binaries would like to enable memory linter in the default configuration.

[flands]

The memory limiter is critical and difficult to enable by default -- agreed something like this is needed and must be in core. Even better if ballast size did not need to be set in two different places.

[tigrannajaryan]

@pavolloffay Can you clarify: percentage of what? Where do we get the total to calculate the fraction?

[pavolloffay]

The total amount of memory would be derived from the host environment. I haven't looked into this closely, but there should be a way to get total memory of the runtime. For example JDK supports this https://www.eclipse.org/openj9/docs/xxinitialrampercentage/

[morigs]

Is there any progress? Is somebody working on this?

[pavolloffay]

I have started working on this.