snakeyaml conflict with Spring Boot 2.0.2.RELEASE and up

[jloisel]

Elasticsearch 6.2.x and up (including 6.3.0) is incompatible with Spring Boot 2.0.2.RELEASE and up due to snakeyaml.

java.lang.NoClassDefFoundError: org/yaml/snakeyaml/LoaderOptions
at org.springframework.boot.env.OriginTrackedYamlLoader.createYaml(OriginTrackedYamlLoader.java:68)
at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:137)
at org.springframework.boot.env.OriginTrackedYamlLoader.load(OriginTrackedYamlLoader.java:75)
at org.springframework.boot.env.YamlPropertySourceLoader.load(YamlPropertySourceLoader.java:50)

Spring Boot uses snakeyaml 1.19 which has a new class LoaderOptions, while elasticsearch uses 1.17, which has not this class.

[tlrx]

The current version of snakeyaml used in Elasticsearch (1.17) depends on the version of the Jackson library (2.8.10 as of today).

Upgrading Jackson to a more recent version (2.9.x) would bring a new version of snakeyaml that contains a performance regression (starting 1.18, see FasterXML/jackson-dataformats-text#67). Also, the last version of Jackson that we tested (2.9.6) as a serious issue with YAML serialization (see FasterXML/jackson-dataformats-text#90) and it's not sure where the cause of the issue is located (snakeyaml or jackson's yaml codec).

For these two reasons we are holding snakeyaml and jackson updates. Also, please note that we do not support Elasticsearch to be embedded and one of the motivation are the classpath/classloader conflicts that can happen.

[jloisel]

👍 Thanks for the insight.

I'm not embedding elasticsearch, i'm simply using the transport client to connect to an external elasticsearch cluster. Sadly this causes a blocking situation where it's impossible to upgrade Spring Boot to get latest fixes.

[jloisel]

Also checked if using high level rest client could solve this issue. Sadly not because it depends on elasticsearch artifact too.

[tlrx]

Sadly not because it depends on elasticsearch artifact too.

More exactly, it depends on Jackson to parse/generate JSON requests and responses. Please don't blame Elasticsearch or the version of Jackson we use because we have good reason to use this specific version.

The issue you have comes from the fact that your project is using two libs (elasticsearch client and spring boot) that have the same dependency but with a conflicting version. This situation could happen with any dependency you have, and you can't expect Elasticsearch and Spring Boot to be in sync or to always be both on the latest version of a dependency. You have to fix this using your dependency management tool or load things into separate classloaders.

[jloisel]

I completely understand, i'm not blaming any of the libraries. I also fully understand the valid reasons why you keep jackson 2.8.10 instead of 2.9.x.

I'll simply freeze the upgrades until the dependency conflict is resolved. I guess you won't stay with Jackson 2.8.x forever and snakeyaml will possibly fix the performance hog seen in latest versions too :)

Jackson / SnakeYaml are such central libraries in many projects that any issue it may introduce has an impact on many projects down the road.

[dadoonet]

@jloisel You can also think about shading your Jackson version may be?
Have a look at this blog post: https://www.elastic.co/blog/to-shade-or-not-to-shade

May be that could give you an idea to solve this?

[jloisel]

It can be an interesting workaround if we really don't find another solution. I'd rather not tweak the build which is quite simple as of now (and relying on spring-boot-maven-plugin), until I absolutely have to.

I like to keep things simple when possible :)

As there is no hurry upgrading as of now, i'll just simply stick with an older Spring Boot version. I'll upgrade everything when the issue is resolved in the next coming months. I'm sure this issue won't be left as is for years.

[kinbiko]

Just leaving a note to any other lost souls that may come across this issue.

I saw exactly this happening when running tests in IntelliJ. However, the tests passed when running with Gradle in the command line.
The difference between the two is the test runner that's being used. One workaround is to change your test runner in IntelliJ to use the Gradle one instead of 'Platform Test Runner'. In your IntelliJ settings, ensure you have the following config:

Build, Execution, Deployment >
	Build Tools >
		Gradle >
			Runner >
				Run tests using: Gradle Test Runner

Granted this is a workaround, not a proper fix.