[BUG] Reactor Netty dependency conflicts with Azure Synapse and Storage SDK: Could not initialize class reactor.netty.http.client.HttpClientConfig

[limotley]

Describe the bug
When submitting an application that uses the blob and data table libraries to a Synapse pool, I get the reactor netty conflict below. I have no issues with dependencies when running the application locally, which leads me to believe this is an issue with the dependencies introduced by the Synapse runtime (see list here). I have looked through these two similar issues: [1] [2], but I can't use the version pinning fix that I used for jackson due to the reactor netty libraries having different major versions, and trying to replace the netty dependency with httpok doesn't help as the Synapse runtime also introduces a version of httpok that produces conflicts. I'm wondering if there's any workaround for this that I haven't thought of, or if my only option is to wait for the Synapse team to put out a runtime with more up to date libraries.

Edit: forgot to mention initially that I am shading the JAR I upload to Synapse as well

Exception or Stack Trace

java.lang.NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientConfig
	at reactor.netty.http.client.HttpClientConnect.<init>(HttpClientConnect.java:83)
	at reactor.netty.http.client.HttpClient.create(HttpClient.java:393)
	at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:119)
	at com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:18)
	at com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58)
	at com.azure.core.http.HttpClient.createDefault(HttpClient.java:50)
	at com.azure.core.http.HttpClient.createDefault(HttpClient.java:40)
	at com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62)
	at com.azure.storage.blob.implementation.util.BuilderHelper.buildPipeline(BuilderHelper.java:135)
	at com.azure.storage.blob.BlobServiceClientBuilder.buildAsyncClient(BlobServiceClientBuilder.java:112)
	at com.azure.storage.blob.BlobServiceClientBuilder.buildClient(BlobServiceClientBuilder.java:93)

Setup (please complete the following information):

• OS: Windows
• IDE: IntelliJ
• Library/Libraries: dependencytree.txt
• Java version: 8
• App Server/Environment: Azure Synapse
• Frameworks: Spark

[mhamilton723]

+1 on this issue I cannot use azure text analytics API (which depends on azure libs for java) on databricks

[alzimmermsft]

We have a quick guide on shading the Azure SDKs here that may be used as a quick attempt to unblock this issue/these issues while a deeper investigation is done.

[limotley]

@alzimmermsft sorry, should've mentioned in my initial post that I am already using a shaded jar. I'm not sure what's going on within Synapse that it can still produce these conflicts

[joshfree]

@pallavit could you please follow up?

[pallavit]

@alzimmermsft sorry, should've mentioned in my initial post that I am already using a shaded jar. I'm not sure what's going on within Synapse that it can still produce these conflicts

Can you please confirm that you are relocating both io.netty and reactor?The link here https://github.com/Azure/azure-sdk-for-java/wiki/Creating-a-Shaded-Jar has details on how to do it.

[limotley]

@pallavit ah good call, I checked and we were missing the relocations. Seems to be working in Synapse now with that added, thanks for your help!

[pallavit]

I am glad it is working for you. I will close the issue but please feel free to reach out in future as needed.