[WIP] Upgrade Karaf from 4.4.6 to 4.4.7

[holgerfriedrich]

This is WIP and prepares for next Karaf version 4.4.7 (currently still at SNAPSHOT level).

Fixes #4283
Fixes CVE in BouncyCastle 1.77

Upgrade Karaf from 4.4.6 to 4.4.7

• Sync runtime dependencies with Karaf 4.4.7, most notably:
  • BouncyCastle 1.78.1
  • CXF 3.6.4
  • Jackson 2.17.2
  • JNA 5.15.0
  • JAXB 2.3.9
  • commons-io 2.17.0
  • commons-lang3 3.17.0
  • XBean 4.25
• Resolve itest runbundles

[andrewfg]

@holgerfriedrich did you see the comments on #3315 about our dependency on Jetty 9.x whereby Jetty 9.x is EOL but upgrading to Jetty 10 or 11 or 12 apparently requires Karaf 4.5 ??

So in other words your incremental Karaf 4.4.x upgrades are all fine and good, but do not resolve the issue that we are over dependendent on Jetty 9.x which is now THREE major versions out of date from current Jetty code.

[J-N-K]

We can't do anything about that. Karaf 4.5 should bring at least Jetty 10, but TBH - looking at the current speed of development - I don't see that before next year.

[andrewfg]

@J-N-K is there a hard point in our Karaf 4.4.x configuration that absolutely forbids importing Jetty 10+ ?? .. I am wondering if there may be some approach whereby we can try to decouple these inter dependencies. The current tight interdependence put us (IMHO) in a very risky situation if one or other of the two code bases is eventually abandoned.

[holgerfriedrich]

@andrewfg the last discussion about Jetty on the Karaf mailing list I have seen is there: https://lists.apache.org/thread/n30xb51gffwmfxs1ldw8zlvj21moszyh

I agree with @J-N-K, it is likely that a Jetty upgrade will not happen soon.
I cannot tell what our options are (besides waiting).

Karaf 4.4.x upgrades are still useful. IIRC with the last upgrades we got ready for Java 21 and closed a relevant CVE in Jetty. 4.4.7 will get bouncy castle to a recent version and upgrade a few more dependencies.

[J-N-K]

I wonder if Karaf is a dead-end. PaxWeb does not even support Jetty 12 (there is a version for Jetty 10 - PayWeb 9.x), but Karaf does not use that.

[andrewfg]

I am still wondering of there is some way to build OH to import Jetty 12 directly into OH core, and exclude the older Jetty parts that would come indirectly via Karaf. That would give us much tighter control over versioning.

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/unifi-binding-beta-3-2-0-3-6-0/131156/183

[J-N-K]

The difficult thing is that we also use both, Jetty server and Jetty client. Since they have common components it would be a lot easier if we use the same version for both. And that's where the problem is: For the server part (especially the servlets) we depend on PaxWeb and that currently doesn't work with the newer versions of Jetty (11/12).

[andrewfg]

Hmm.

It seems to me that we are in some jeopardy here..

• OH depends on Karaf
• OH depends (therefore) on Paxweb
• OH depends (therefore) on Jetty
• Current Paxweb (and therefore) Karaf use Jetty 9.x
• Jetty 9.x is EOL and THREE major versions behind the current 12.x
• Paxweb dev is resisting upgrading the Jetty dependency from 9.x to 12.x due to a) the amount of effort, and b) the risk of breaking changes
• Karaf dev is (therefore) considering dropping Paxweb because (as mentioned above) it is not current
• Karaf dev is (therefore) considering switching from Jetty to (say) Tomcat

It seems to me that OH architecture council needs to do some thinking. Do we accept our core to be so dependent on above mentioned weak points in dependency support? And/or do we have a vision of the future path?

Perhaps we need to contribute active PRs to Paxweb and (therefore) Karaf in order to bypass the bottleneck? And/or perhaps we need to make our own fork of Paxweb (and eventually Karaf)?

[kaikreuzer]

Paxweb dev is resisting upgrading the Jetty dependency from 9.x to 12.x due to a) the amount of effort, and b) the risk of breaking changes
Karaf dev is (therefore) considering dropping Paxweb because (as mentioned above) it is not current
Karaf dev is (therefore) considering switching from Jetty to (say) Tomcat

Do you have links to these discussions in the respective communities? First step might be to involve ourselves there.
A decision by Paxweb not to upgrade essentially means that they have given up the project...

[andrewfg]

Do you have links to these discussions in the respective communities?

@kaikreuzer see links below; note: you need to read it all, including opening the quoted text bubbles..

• The Karaf discussion (already provided by @holgerfriedrich) is here
• It references a prior discussion on Paxweb in Karaf here

[J-N-K]

I don't think that is what @grgrzybek said about future support. The last discussion regarding Jetty 11 (as Pax Web 10) that I remember is https://lists.apache.org/thread/4lwbt9g0w707zomnq6cfx179n8w90d3h: It's not ready (and as far as I can see there is still no release for Pax Web 10), but there is work on that. I can't find any comment about not supporting Jetty 12.

Changing to Jetty 11 (or 12) is also a big step for us because we need to move from javax to jakarta namespace. We should also upgrade JAXB to version 4 then.

[grgrzybek]

Yes - moving from javax to jakarta is a huge effort - that's why Pax Web 10 is still to be done...
And while WAR/WAB and even Whiteboard (which implies HttpService underneath) are actually ready, entire Pax Web 10 has other JavaEE deps to switch to Jakarta (like CDI or JSF) which is more problematic because CDI/JSF OSGi support is modest to say the least...

And because I feel that Pax Web 10 would be much less than Pax Web 8/9 (9 differs only by switching Jetty 9 to Jetty 10, which are still javax.servlet), I intentionally delay it until I have higher motivation (and more time).

And yes - Pax Web 10 is using Jetty 12 (but not in it's multi-servlet-api extent) - only jakarta.servlet package is supported.
Ideally Pax Web 10 should ship with Whiteboard tracker for javax.servlet based Whiteboard services that could be wrapped with jakarta.servlet API (servlets, filters, listeners, error pages, welcome files, JSPs, SCIs).

[andrewfg]

I don't think that is what @grgrzybek said about future support.

Hmm. You may be right. He certainly has added support for Jetty 12 at least on this repo .. but I am not sure if that is the branch on which Karaf 4.5 would (eventually) be based ??

[J-N-K]

Pax Web 9 doesn't solve our issues with outdated Jetty (as Jetty 10 is also EoL for community support), though.

[grgrzybek]

My work on future Pax Web 10 based on Tomcat 10 (soon - 11), Undertow 2.3 and Jetty 12 is here: https://github.com/ops4j/org.ops4j.pax.web/commits/main-jakarta/

And I am aware that if Karaf 4.5 needs JakartaEE 10 support for web-related OSGi CMPN specifications (https://docs.osgi.org/specification/osgi.cmpn/8.1.0/) then either Felix HTTP is needed as default implementation or work on Pax Web 10 has to be hastened...

Mind that OSGi CMPN 8.1 is quite relentless for web-related specifications:

• chapter 102: HttpService is gone
• chapter 128: Web Applications specification is still JavaEE
• chapter 140: Whiteboard specification is done and renamed

[shikousa]

@grgrzybek Thanks for the information. It seems that for the Unifi binding in Openhab we need partitioned cookies. The current Pax Web 8/Jetty 9 does not support the setPartitioned() feature. Is there some kind of workaround available or do we need to wait for >= Pax Web 9/Jetty 10 in Karaf?

https://community.openhab.org/t/unifi-binding-beta-3-2-0-3-6-0/131156/180

[grgrzybek]

Oh, I wasn't aware of partitioned cookies...
Jetty added an attribute in jetty/jetty.project@f82844e
I think not all browsers (Safari?) support it and definitely it's not part of Servlet API.
So please don't expect a unified (Jetty / Tomcat / Undertow) Pax Web support anytime soon...

[J-N-K]

Oh, I wasn't aware of partitioned cookies... Jetty added an attribute in jetty/jetty.project@f82844e I think not all browsers (Safari?) support it and definitely it's not part of Servlet API. So please don't expect a unified (Jetty / Tomcat / Undertow) Pax Web support anytime soon...

I believe we need that on client side, but I vaguely remember that using Jetty 9 server and Jetty 10 client in the same OSGi container is difficult. I might be wrong on that.

[shikousa]

@grgrzybek I think partitioned cookies are already supported in Jetty 10. So isn't it possible that Pax Web 9 also supports it?

[grgrzybek]

@grgrzybek I think partitioned cookies are already supported in Jetty 10. So isn't it possible that Pax Web 9 also supports it?

May I ask you please to create an issue here? https://github.com/ops4j/org.ops4j.pax.web/issues

I may look at it, as it sounds interesting and important ;) But I can't promise any release date with this implemented... Please provide some context and links there.

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/unifi-and-2fa/107352/19