Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move client classes and dependencies out from websocket-core #2174

Closed
joakime opened this issue Jan 31, 2018 · 17 comments · Fixed by #4881
Closed

Move client classes and dependencies out from websocket-core #2174

joakime opened this issue Jan 31, 2018 · 17 comments · Fixed by #4881
Assignees

Comments

@joakime
Copy link
Contributor

joakime commented Jan 31, 2018

Currently, websocket-core has client classes and dependencies.

These are not appropriate for websocket-core as they contaminate the dependency tree and complicate the WebAppClassloader for pure websocket servers. (currently Jetty Native WebSockets, not JSR356, and the future proposed Reactive streams server)

These classes, test cases, and dependencies should be moved out of websocket-core

@joakime joakime self-assigned this Jan 31, 2018
@gregw
Copy link
Contributor

gregw commented Feb 1, 2018

I remain unconvinced we need this complexity. Simplicity is worth a lot an needless fine grained modules can create a lot of complexity and confusion.

  • The client/server packages within core are only 40kB/44kB uncompressed, so I do not think size is an issue.
  • The extra dependencies are simple: just jetty-client or jetty-server and both are marked as provided, so they are not hard dependencies.
  • jetty-core is never intended to be used directly, but always as a transitive dependency via an API layer that can make any required dependencies hard.
  • there are always classes within jars that are not used and I think it is needless to try to decompose many many little jars just to try to save a few 40kB of memory that wont actually be used unless the classes are loaded.

In fact, I think we could go the other way and remove/merge modules such as websocket-server and websocket-servlet, even websocket-common is a little dubious.

I think for each foobar API we should have foobar-websocket-common, foobar-websocket-server, foobar-websocket-client, foobar-websocket-test. Everything else should either be in core or the foobar commons.

@jmcc0nn3ll
Copy link
Contributor

I had Joakim walk me through this a bit earlier because something doesn't smell right, why should this websocket goop be any different then the rest of Jetty structure wise? You have a jetty-client that depends on jetty-io and has no concept of the servlet-api or the jetty-server itself, you have a jetty-server that depends on jetty-io and no concept of a jetty-client but does care about the servlet-api and stuff like that. We have bent over backwards historically to ensure a clean separation of concerns exists and this seems like websocket is something different then just a protocol with a client and a server and some shared io classes

I am not seeing what benefit there is to glomming everything together into a -core module (which is a new term?) that tries to be all things to all people.

IMO:

  • websocket-core should be renamed to websocket-io and exist as jetty-io does, actually just a layer on top of that
  • websocket-server should be everything required to run server or be a dependency for *-server APIs
  • I don't see the point of a websocket-servlet artifact, that should be rolled into websocket-server
  • websocket-client should be everything required to run a client and have no knowledge of server related things, and be the basis for client API's
  • javax.websocket can be the carnage based API that tries to glom things together and be all things for all people but at least up to that point we have followed the basic conventions the rest of Jetty has followed forever. In fact it feels like making a glommed together core is in response to their decisions as opposed to ours?

I can see how it got to this point out of convenience for development which is what you have been advocating for months, don't get hung up on the project structure, focus on API's, but once you guys have the refactor fleshed out I would think a good test of that would be at least validating separation of concerns like the other protocol modules in Jetty.

Unless I am missing something fundamental here?

@gregw
Copy link
Contributor

gregw commented Feb 1, 2018

@jmcc0nn3ll @joakime

I can live with websocket-io as a better name than websocket-core.

But having a few client/server specific classes in websocket-io is not unusual and more or less follows what we have with jetty-io and jetty-http, both of which have classes and methods that are only of use to clients and/or servers:

  • jetty-io contains NegotiatingClientConnection*, ClientConnectionFactory and SslClientConnectionFactory, which are client side only and equivalent to the websocket handshaking stuff in websocket-io
  • jetty-http contains PathMap, CookieCutter etc. which are server side only

We don't have jetty-io-client, jetty-io-server, jetty-http-client, jetty-http-server, so why should we have websocket-io-client and websocket-io-server? Specially when they are only 3 or 4 classes in IO.
The only real difference is that because websocket is higher up the food chain, it has dependencies that are client and server specific.

But also note that things like java.net are not split out into java.net.client java.net.server` even though there are client and server specific classes!

I agree that websocket-servlet and websocket-server should be merged, but they are API specific, so we cannot merge in the server parts from core/io. Instead they should be merged to jetty-websocket-server, which currently only has 1 class (like websocket-server!!!).

I think that websocket-common should go and anything need in there should go to io or one/both of the base API modules.

In summary, I think the hierarchy should be nice and simple:

                 websocket-io
                   ^       ^
                  /         \
                 /           \
                /             \
           jetty-             javax-
           websocket          websocket
           ^    ^                ^   ^
          /      \              /     \
         /        \            /       \
        /          \          /         \
  jetty-        jetty-      javax-      javax-
  websocket-    websocket-  websocket-  websocket-
  client        server      client      server

And finally, as I keep saying... perhaps once we are done a requirement may emerge that will require us to change how we decompose the impl into modules. However, no such requirement is currently apparent, so if one does emerge, any preemptive decomposition is likely to be wrong.

@jmcc0nn3ll
Copy link
Contributor

jmcc0nn3ll commented Feb 1, 2018

@gregw , in theory it looks nice and straight forward and so long as the dependencies are not trying to push a mess of <excludes> and <provides> magic onto users that is fine. I stopped with @joakime when I saw a mess of ugly osgi includes and excludes that i believe were automagically generated from the maven build. Actually, if you updated your graphic with the intended jetty-* dependencies that would help, like websocket-io should only depend on jetty-io in my mind, and in turn jetty-util.

@gregw
Copy link
Contributor

gregw commented Feb 1, 2018

pushing my ascii-art foo to its max:

                   jetty-util
                       ^
                       |
                   jetty-io
                       ^
                       |
                   jetty-http
                   ^   ^   ^    servlet-api
                  /    |    \       ^
                 /     |     \      |
                /      |      \     |
     jetty-client      |      jetty-server
       ^   ^   ^       |       ^   ^    ^
      /     \   .      |      .   /      \
     /       \   .     |     .   /        \
    |         \   .    |    .   /          |
    |          \ websocket-io  /           |
    |           \  ^       ^  /            |
    |            \/         \/             |
    |            /\         /\             |
    |           /  \       /  \            |
    |     jetty-    |     |   javax-       |
    |     websocket |     |   websocket    |
    |     ^  ^      |     |      ^   ^     |
    |    /    \    /       \    /     \    |
    |   /      \  /         \  /       \   |
    |  /        \/           \/         \  |
  jetty-       jetty-      javax-      javax-
  websocket-   websocket-  websocket-  websocket-
  client       server      client      server                                                                                                             

dotted lines are provided dependencies

@jmcc0nn3ll
Copy link
Contributor

Man, I don't like provided dependencies at all for an io library like that. Is there that much code in there that you can't just put client and server dependencies in the end oriented modules and do away with jetty-websocket and javax-websocket?

I don't see why there needs to be circles in there at all, purity wise it should be jetty-http -> websocket-io then branch to [ws-client | ws-server] and then javax-websocket-client builds on top of that ws-client and then javax-websocket-client is a dependency on javax-websocket-server....that way it is all pure straight lines and makes good maven sense (and osgi sense).

@jmcc0nn3ll
Copy link
Contributor

  • websocket-io
    -- jetty-http

  • ws-client
    -- websocket-io
    -- jetty-client

  • ws-server
    -- websocket-io
    -- jetty-server

  • javax-websocket-client
    -- ws-client

  • javax-websocket-server
    -- ws-server
    -- javax-websocket-client

Maybe ws-client and ws-server have the stuff needed to act as the jetty specific api version in them as well and it is just ignored when being used in the javax context.

@stale
Copy link

stale bot commented Nov 20, 2019

This issue has been automatically marked as stale because it has been a full year without activit. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the Stale For auto-closed stale issues and pull requests label Nov 20, 2019
@gregw gregw removed the Stale For auto-closed stale issues and pull requests label Nov 20, 2019
@gregw
Copy link
Contributor

gregw commented Nov 20, 2019

@lachlan-roberts can this be closed? ie are we happy with the current package structure?

@joakime
Copy link
Contributor Author

joakime commented Nov 20, 2019

Build a minimal websocket client only project. (you should have no server dependencies)
Figure out what's needed to configure it's <dependencies>.
If it's tedious, or problematic, or fraught with gotchas, then this issue still has relevance.

@gregw
Copy link
Contributor

gregw commented Nov 20, 2019

@joakime I don't get it. If you want to know the dependencies for a core client:

gregw@pellet: ~/src/jetty-10.0.x/jetty-websocket/websocket-core (jetty-10.0.x)
[2049] mvn dependency:tree
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------< org.eclipse.jetty.websocket:websocket-core >-------------
[INFO] Building Jetty :: Websocket :: Core 10.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ websocket-core ---
[INFO] org.eclipse.jetty.websocket:websocket-core:jar:10.0.0-SNAPSHOT
[INFO] +- org.eclipse.jetty:jetty-util:jar:10.0.0-SNAPSHOT:compile
[INFO] +- org.eclipse.jetty:jetty-io:jar:10.0.0-SNAPSHOT:compile
[INFO] +- org.eclipse.jetty:jetty-http:jar:10.0.0-SNAPSHOT:compile
[INFO] +- org.eclipse.jetty:jetty-xml:jar:10.0.0-SNAPSHOT:compile (optional) 
[INFO] +- org.eclipse.jetty:jetty-client:jar:10.0.0-SNAPSHOT:provided
[INFO] |  \- org.eclipse.jetty:jetty-alpn-client:jar:10.0.0-SNAPSHOT:provided
[INFO] +- org.eclipse.jetty:jetty-server:jar:10.0.0-SNAPSHOT:provided
[INFO] |  \- org.eclipse.jetty.toolchain:jetty-servlet-api:jar:4.0.2:provided
[INFO] +- org.eclipse.jetty.toolchain:jetty-test-helper:jar:5.3:test
[INFO] |  \- org.hamcrest:hamcrest:jar:2.1:test
[INFO] \- org.junit.jupiter:junit-jupiter:jar:5.5.1:test
[INFO]    +- org.junit.jupiter:junit-jupiter-api:jar:5.5.1:test
[INFO]    |  +- org.apiguardian:apiguardian-api:jar:1.1.0:test
[INFO]    |  +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO]    |  \- org.junit.platform:junit-platform-commons:jar:1.5.1:test
[INFO]    +- org.junit.jupiter:junit-jupiter-params:jar:5.5.1:test
[INFO]    \- org.junit.jupiter:junit-jupiter-engine:jar:5.5.1:test
[INFO]       \- org.junit.platform:junit-platform-engine:jar:1.5.1:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.172 s
[INFO] Finished at: 2019-11-21T10:16:29+11:00
[INFO] ------------------------------------------------------------------------

or for a javax client:

gregw@pellet: ~/src/jetty-10.0.x/jetty-websocket/javax-websocket-client (jetty-10.0.x)
[2047] mvn dependency:tree
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------< org.eclipse.jetty.websocket:javax-websocket-client >---------
[INFO] Building Jetty :: Websocket :: javax.websocket :: Client Implementation 10.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ javax-websocket-client ---
[INFO] org.eclipse.jetty.websocket:javax-websocket-client:jar:10.0.0-SNAPSHOT
[INFO] +- org.eclipse.jetty.websocket:javax-websocket-common:jar:10.0.0-SNAPSHOT:compile
[INFO] |  \- org.eclipse.jetty.websocket:websocket-core:jar:10.0.0-SNAPSHOT:compile
[INFO] +- org.eclipse.jetty:jetty-client:jar:10.0.0-SNAPSHOT:compile
[INFO] |  +- org.eclipse.jetty:jetty-http:jar:10.0.0-SNAPSHOT:compile
[INFO] |  +- org.eclipse.jetty:jetty-io:jar:10.0.0-SNAPSHOT:compile
[INFO] |  \- org.eclipse.jetty:jetty-alpn-client:jar:10.0.0-SNAPSHOT:compile
[INFO] +- org.eclipse.jetty.toolchain:jetty-javax-websocket-api:jar:1.1.1:compile
[INFO] +- org.eclipse.jetty:jetty-xml:jar:10.0.0-SNAPSHOT:test
[INFO] |  \- org.eclipse.jetty:jetty-util:jar:10.0.0-SNAPSHOT:compile
[INFO] +- org.eclipse.jetty.toolchain:jetty-test-helper:jar:5.3:test
[INFO] |  \- org.hamcrest:hamcrest:jar:2.1:test
[INFO] \- org.junit.jupiter:junit-jupiter:jar:5.5.1:test
[INFO]    +- org.junit.jupiter:junit-jupiter-api:jar:5.5.1:test
[INFO]    |  +- org.apiguardian:apiguardian-api:jar:1.1.0:test
[INFO]    |  +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO]    |  \- org.junit.platform:junit-platform-commons:jar:1.5.1:test
[INFO]    +- org.junit.jupiter:junit-jupiter-params:jar:5.5.1:test
[INFO]    \- org.junit.jupiter:junit-jupiter-engine:jar:5.5.1:test
[INFO]       \- org.junit.platform:junit-platform-engine:jar:1.5.1:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.279 s
[INFO] Finished at: 2019-11-21T10:16:19+11:00
[INFO] ------------------------------------------------------------------------

There are classes in those dependencies that will not be used and splitting core into more packages won't change that. If you really want an absolute minimal set of classes, then some dynamic analysis is required that will not be helped by more core packages?

@joakime
Copy link
Contributor Author

joakime commented Nov 20, 2019

The last time I tried, there were more lines of <dependency><exclusions> then there were <dependency>.
And that level of nonsense made gradle and other build libraries (sbt) complain that there were too many exclusions and you should just declare the dependency without transitive enabled.

@joakime
Copy link
Contributor Author

joakime commented Nov 20, 2019

There was also a bunch of asm errors from libraries that did their own bytecode scanning when encountering features in websocket-core that are impossible without the server classes present.

@gregw
Copy link
Contributor

gregw commented Nov 20, 2019

I'm just not understanding the problem? Just use core dependency with no exclusions. OK it does depend on jetty-server, which is probably not really needed... but there are always going to be unused classes in the classpath unless you do a dynamic analysis and trim to specific classes. I thinking adding packages is just too much unnecessary complexity to save a few 100kB of jars

@joakime
Copy link
Contributor Author

joakime commented Nov 21, 2019

The design of websocket-core is just antithetical to how to properly create a jar in maven, osgi, and java 9+ (JPMS).
The whole "optional" hard dependencies in websocket-core is the reason. (hard dependencies on jetty-client and jetty-server that are hidden by sleight of hand and trickery)
Yes, lots of deployed maven jars have optional behavior, but not via hard dependencies (like import statements).

The reason this issue exists is to fix that.
Take client and server out of core.
Make them their own jars.

jetty-http
jetty-io
jetty-util
   websocket-core
      javax-websocket-common
      jetty-websocket-common
      websocket-core-client
         jetty-websocket-client
         javax-websocket-client
      websocket-core-server
         jetty-websocket-server
         javax-websocket-server

Then there's the maintenance angle as well.
With the colocation of both jetty-server and jetty-client in the same project (websocket-core) it's trivially easy to accidentally pull in dependencies from the opposite side, a mistake you won't notice until you do something outside of the project to use that dependency with the cross contamination.
With a properly separated project you cannot do that.

@gregw
Copy link
Contributor

gregw commented Nov 21, 2019

I still don't get the strong concern.
Optional/provided dependencies are features of maven so I can't see how they are antithetical to them. They are not tricks or sleight of hand.
There are many such dependencies in jetty and the fact that these ones cross a client/server divide is not that significant to the build. The fact they are used in imports is no less breaking than if they are used via reflection by a class that is instantiated and depends on a optional/provided dependency.

We have split the packages into client/server within the module, so this should be pretty trivial to implement. Let's see a PR and see if the extra complexity is worth it.

@lachlan-roberts lachlan-roberts added this to the 10.0.x milestone Jan 30, 2020
@lachlan-roberts
Copy link
Contributor

I'm fine with the current websocket-core structure, but will leave this and #2173 open for discussion.
It is not so important for a 10.0.0 release so I have removed it from that project.

@sbordet sbordet removed this from the 10.0.x milestone Feb 4, 2020
lachlan-roberts added a commit that referenced this issue May 12, 2020
…modules

Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
lachlan-roberts added a commit that referenced this issue May 25, 2020
Issue #2173 & #2174 - separate out client and server parts of websocket-core
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants