Limit memory usage of RocksDB and make it more configurable

[nscuro]

Description

Limit memory usage of RocksDB and make it more configurable.

Mostly following the guide from Confluent (https://docs.confluent.io/platform/current/streams/developer-guide/memory-mgmt.html). Per default, Kafka Streams creates a RocksDB block cache of up to 50MiB for each state store, which quickly adds up. Since RocksDB memory is off-heap, keeping it contained is a challenge.

With this change, we configure a single block cache to be used for all stores in a Kafka Streams instance. We limit the maximum cache size to 128MiB per default, but provide the ability to configure that if necessary.

Addressed Issue

N/A

Additional Details

N/A

Checklist

• I have read and understand the contributing guidelines
• This PR fixes a defect, and I have provided tests to verify that the fix is effective
• This PR implements an enhancement, and I have provided tests to verify that it works as intended
• This PR introduces changes to the database model, and I have updated the migration changelog accordingly
• This PR introduces new or alters existing behavior, and I have updated the documentation accordingly

[codacy-production]

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ +2.21% (target: -1.00%)	❌ 0.00% (target: 70.00%)

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (232adcc)	172	140	81.40%
Head commit (d8946c3)	6892 (+6720)	5762 (+5622)	83.60% (+2.21%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#1498)	34	0	0.00%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

_{Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more}

[nscuro]

RocksDbConfigSetter can not be reasonably tested without excessive mocking. Tested manually.