kafka: Reduced maximum log line size to 128KiB

[michael-redpanda]

Fixes: CORE-2409

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v23.3.x
• v23.2.x

Release Notes

Bug Fixes

• Reduces maximum log line size from 1MiB to 128KiB to reduce occurrences of memory allocation failures

[emaxerrno]

this is cool. wondering if there is a global thing we should do anyway. like is there a seastar setting we set in main.cc for all loggers to 128KiB default max msg size.

[michael-redpanda]

this is cool. wondering if there is a global thing we should do anyway. like is there a seastar setting we set in main.cc for all loggers to 128KiB default max msg size.

I don't think there is a knob in seastar for that. Everything ends up in a contiguous char[] buffer. We could go through and apply this truncating logger to all of our loggers.

[vbotbuildovich]

/backport v23.3.x

[vbotbuildovich]

/backport v23.2.x

[emaxerrno]

I don't think there is a knob in seastar for that. Everything ends up in a contiguous char[] buffer. We could go through and apply this truncating logger to all of our loggers.

let's do that, because this will likely happen in other subsystems, for example i imagine the admin api here would be vulnerable.