java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer when enabling topic metadata compression

[codelipenghui]

Describe the bug
We have introduced managed ledger metadata compression in #11490, but we will get java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; when run Pulsar on JDK1.8.

17:36:54.799 [mock-pulsar-bk-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught 
java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
 at io.airlift.compress.zstd.ZstdCompressor.compress(ZstdCompressor.java:101) ~[aircompressor-0.19.jar:0.19]
 at org.apache.pulsar.common.compression.CompressionCodecZstd.encode(CompressionCodecZstd.java:75) ~[pulsar-common-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
 at org.apache.bookkeeper.mledger.impl.MetaStoreImpl.compressLedgerInfo(MetaStoreImpl.java:321) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
 at org.apache.bookkeeper.mledger.impl.MetaStoreImpl.asyncUpdateLedgerIds(MetaStoreImpl.java:134) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
 at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$null$1(ManagedLedgerImpl.java:491) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
 at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[managed-ledger-2.8.1-SNAPSHOT.jar:2.8.1-SNAPSHOT]
 at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [bookkeeper-common-4.14.1.jar:4.14.1]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_281]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_281]
 at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.66.Final.jar:4.1.66.Final]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_281]
17:36:58.356 [pulsar-load-manager-120-1] INFO  org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Writing local data to metadata store because maximum change 59.76470708847046% exceeded threshold 10%; time since last report written is 5.02 seconds
17:37:11.277 [TestNG-method=testRestartBrokerEnableManagedLedgerInfoCompression-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [prop/ns-abc/topic-5d9785f202336c4c] [test] Closed Consumer (not connected)
17:37:11.277 [TestNG-method=testRestartBrokerEnableManagedLedgerInfoCompression-1] INFO  org.apache.pulsar.client.impl.ProducerImpl - [prop/ns-abc/topic-5d9785f202336c4c] [test-1-0] Closed Producer (not connected)
!!!!!!!!! FAILURE-- [TestClass name=class org.apache.pulsar.broker.service.ManagedLedgerCompressionTest].testRestartBrokerEnableManagedLedgerInfoCompression([])-------

For branch-2.8 and master, build on JDK8 or JDK11, run on JDK8 can works.
For branch-2.8 and master, build on JDK8 or JDK11, run on JDK11 can works.

But for branch-2.7, both are works.

After investigating the issue, it's related to the aircompressor upgrade, we have upgraded aircompressor from 0.16 to 0.19 #10983. After upgrade the aircompressor version for branch-2.7, the error appears.

Looks like related to the aircompressor, maybe need to add --release 1.8 for aircompressor.

The producer also using the aircompressor to compress data, but after the test on both branch-2.7, branch-2.8, and master, it works, no error happens. The difference is the producer run into this branch

pulsar/pulsar-common/src/main/java/org/apache/pulsar/common/compression/CompressionCodecZstd.java

Line 65 in 6704f12

compressedLength = ZStdRawCompressor.compress(

but the managed ledger metadata compression run into here

pulsar/pulsar-common/src/main/java/org/apache/pulsar/common/compression/CompressionCodecZstd.java

Line 72 in 6704f12

ByteBuffer sourceNio = source.nioBuffer(source.readerIndex(), source.readableBytes());

To Reproduce
Steps to reproduce the behavior:

1. Build the master branch
2. Change managedLedgerInfoCompressionType=ZSTD in broker.conf
3. Start the standalone
4. See error

[lhotari]

This looks like a bug in aircompressor library. it's not specifying the release configuration for maven compiler plugin as described in https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/ .

In Pulsar this was solved with #10343 .

[merlimat]

This was fixed in #11790