Re-Open the "2.0" discussion

[gbrail]

I'd like to propose that we re-consider a backwardly-incompatible 'Rhino 2.0' project as a few of you proposed before. I used to be against this, but I'm coming around.

For example, things I and others have proposed would make this project a lot more streamlined:

Ideas that make it easier to maintain without necessarily breaking compatibility:

• Require Java 11 minimum (probably Java 17 in a year or so)
• Restructure the source code to modern Gradle standards and into proper Java modules
• Get rid of code designed for backward compatibility with older Java versions
• Get rid of the security manager, which is also coming out of Java

Ideas that would affect compatibility, but not necessarily for everyone:

• Rip out the "language version" capability. This would break many tests and old code, but it would simplify a lot, and reduce the number of times a naive new user selects an old version of Rhino. This would also make Rhino like every other JavaScript engine, in which each version increases support for new language features, and projects not ready to consume them can always use an older version. This would be incompatible but I bet that most projects would work with only minor changes.
• Track down any remaining usage of synchronization and get rid of it. A Rhino "context" would then only be able to be used by one thread at a time. (We might need some thread-safety changes so a single JVM can run many contexts in different threads, however.) This would again align Rhino with every other JavaScript engine.
• Get rid of serialization, which is increasingly seen as a Java mistake and a huge security hole. Unfortunately, this would break continuations, so we might need to find another way.

Ideas that would require anyone embedding Rhino to change their code:

• Change the interface to Scriptable so that it is simpler and supports current-century Java features like iterators.
• Consider rationalizing String, Symbol, and integer property keys into a single, simpler interface
• Consider why we use Scriptable for scopes (which parents but not prototypes) and objects (which have prototypes but not parents) and if changing that could streamline the code or even make it faster.

I may be able to find more time over the next months to work on this, a little bit every week, but I'm a lot more interested if we can go faster!

I'm sure that you all have more ideas!

As for how to do this, we could:

• Use the new "rhino" GitHub org as the main branch for this, leaving org.mozilla.rhino for the pre-2.0 codebase
• Or, create a "rhino2" branch
• Or, create a new project that's called something else

Finally, I'm NOT proposing writing a brand-new JavaScript engine in Java -- the Nashorn team already did that!

[p-bakker]

Also see #1362 and specifically #1362 (comment)

And also the v2 project created a while ago: https://github.com/mozilla/rhino/projects/2

[p-bakker]

I'm all for doing whatever it takes to move the Rhino project forward and making it future proof and more easily maintainable, even if that means introducing breaking changes (as long as we're very good at communicating what we're breaking and how to resolve the breakage). Do think that we have to try to bundle all the 'breaking' we want to do in a single release, to anyone using Rhino only need to go through a breaking upgrade once.

* HOW

As for the how: my preference would be to keep the backwards compatible and a new non-backward compatible version (lets call then v1.x and v2 resp. for now) in the same repo. I would propose to create a new main branch for v2 and create some other branch for the (continued) development of the backwards compatible 1.x version, with a clear understanding that that 1.x version is in maintenance mode

* Scriptable changes

Wrt Change the interface to Scriptable: I assume you're suggesting to make changes to the Scriptable interface? Not sure what kind of changes you're suggesting and whether those could be made in a backward compatible way, but while trying to fix (some of) the issues with the EcmaScript strict mode implementation in Rhino, I also ran into the need to change the Callable interface, see all comments on #442

* Restructure the source code to modern Gradle standards and into proper Java modules

Think we should try to get closure on at least the last 15 or so PR's (of the total of 29 currently open) and do a shout-out to the community to request any changes that aren't submitted as PR's yet to be submitted asap, as to minimize the impact of such relayout of the codebase

* Getting rid of synchronization

Should really discuss in the already existing issue, but we do need to think about the overd of Context creation, if a Context can only be used by a single Thread at a time., as it's might be a scaling issue (also an issue in Graal). Also wonder if/how the Secure EcmaScript (SES) proposal might fit into this: given that the overhead of setting up a Context lies in creating all the standard objects (assuming here) and that in SES those Objects are locked down before being shared in different Compartments, making Compartments a lot more light-weight, could different Compartments be used in different threads concurrently?

* Getting rid of serialization

Personally no problems here, have not seen any other usage of serialization but for Continuations, so if we can offer an alternative there...

In #1178 (comment) and #704 (comment) I think there are already pointers to use Continuations without reliance on Java Serialization (but I might be wrong)

* Rip out the "language version" capability

I'm not sure what I think about this proposal, think we might need to keep at least some versioning system until such time that we are EcmaScript standard compliant.

And even then, we might want to have some flags to allow implementing EcmaScript proposals that aren't yet finalized

Just note that Graal does support specifying language version.

* Re: Ideas that would require anyone embedding Rhino to change their code: & I'm sure that you all have more ideas!

A lot of ideas/suggestions/thoughts have already been collected in the existing v2 project

@gbrail great initiative!

[tonygermano]

I think the context flags can still exist (for example, to enable experimental features) even if the language version is removed. I think this just means that we drop legacy support for non-standard features (though, I'll still fight for e4x) and always present the closest implementation to the current standard as possible by default, with some behavior modifiable through context flags.

I think we should also look at making a clear distinction between public API and internal classes which are subject to change without notice. Right now a lot of what is preventing things being fixed is that embedders have unlimited access to all classes, and so any change could be a breaking change.

[nmaguiar]

I would also join the "fight", like @tonygermano, for e4x (big fan of it myself), as a non-standard experimental feature controlled with context flags or similar on v2.

The main focus should be to keep the base implementation near the standard as much as possible and, as stated by @p-bakker, be very good in stating what is going to be broken and how to solve it while trying to keep all the 'breaking' in one single moment/version.

But there should be a parallel effort to keep so of the most used non-standard features also in v2 and not stuck "to die" on a maintenance v1.x without alternatives. I feel some of them help to keep and bring new usage cases for Rhino overtime.

[p-bakker]

Don't think E4X is on the list of things to be considered to remove. I think it's one of the things that make Rhino unique and stand out from other implementations

[rPraml]

As we use Rhino with some extensions in my company, we would apprecate, if this project gets some progress.

I'd like to propose that we re-consider a backwardly-incompatible 'Rhino 2.0' project as a few of you proposed before. I used to be against this, but I'm coming around.

Great, there are some PRs from me, that will break the API, maybe that's my chance to get them merged ;)

For example, things I and others have proposed would make this project a lot more streamlined:
Ideas that make it easier to maintain without necessarily breaking compatibility:
* Require Java 11 minimum (probably Java 17 in a year or so)

FYI: We currently use Java 11 and it is planned to switch to 17 the next 2-3 monts

* Get rid of the security manager, which is also coming out of Java

+1

* Rip out the "language version" capability.

I think this is not a problem for ME, because we tried to use the latest version

* Track down any remaining usage of synchronization and get rid of it. 

We use one shared scope (#826) here we would need some synchronization. But the javascript executions is then done in a single thread.

* continuations

We do not use continuations

Ideas that would require anyone embedding Rhino to change their code:

This should be no probem here. I could imagine helping with testing or even provide fixes.

Roland

[tuchida]

What are Rhino's strengths that Nashorn and GraalJS do not have? For example, if there are a lot of use cases on Android (since we don't use invokedynamic), why not focus on that?

[p-bakker]

Nashorn is deprecated, Rhino is not :-)

GraalJS is way more EcmaScript compliant, more raw performance, but harder to embed (from what I hear, no personal experience)

Rhino:

• supports continuations
• supports E4X
• (more) focus on embeddability
• specific features to improve scaling (sealed/shared/dynamic scopes for example)
• runs on Android (GraalJS does not)
• IDE parser mode
• ....?

For completeness sake: what Rhino lacks:

• (better) EcmaScript compatibility
  • strict mode
  • const
• Missing (big) EcmaScript features
  • Class support
  • EcmaScript module support
  • async/await
  • several Regex features
  • Ecma402 / Internationalization API impl.

As for your suggestion to focus on Android: I personally rather not, as I think it's a niche in the Rhino usage and engagement of Android App Developers that utilize Rhino has been fairly limited to date

[commonsuppliz]

Rhino also can run on dotnetcore 3.1 and above (windows/macos/linux) thanks to IKVM Revived Project (https://github.com/ikvmnet/ikvm), while nashorn and GraalJS can not run on dotnetcore.

[p-bakker]

Interesting, any reason why someone would want to use Rhino over something like https://github.com/sebastienros/jint on .Net?

[commonsuppliz]

Performance of Rhino on dotnetcore is better than Jint (https://github.com/sebastienros/jint) while there is some overhead by using IKVM. I tried some other .net javascript project but Rhino's performanace is better. While Rhino on java sunspider benchmark is around 1300, Rhino on dotnetcore approximately scores 4500.

[delchev]

We use GraalJS in Eclipse Dirigible since a few years now and are quite happy with the compatibility with the latest ECMA specifications. It also provides a debug functionality exposed as V8 inspector protocol, so Chrome DevTools can connect to it. However, Rhino has its strengths as well...

[p-bakker]

Given that you are apparently using both, what would you say are the strengths of Rhino?

[delchev]

Actually we have used all four - Rhino, Nashorn, V8 and finally GraalJS, in this order and for a while all together.

• Rhino - the first, mature and pure Java implementation. Easy to use and embed in standalone apps. Large users base and lots of samples everywhere. Good performance, except on Android. Comparatively easy to integrate debug protocol. Main drawback for us is the compatibility with the recent ECMA specs.
• Nashorn - was interesting attempt to build more modern JavaScript engine, but had its known issues. We kept it for a while just in case.
• V8 the native one with all the benefits, which everybody knows. The main drawback was in the integration with Java and the performance due to transfers between stacks and the synchronisation of the two object trees. Actually we built this integration by ourselves and it is still somewhere in the old branches. Now it is deprecated.
• GraalJS solved our problem - provided the modern ECMA compatibility, gave comparable performance with the native ones and it is still Java implementation.

If I would suggest what I expect from Rhino 2.0 - pure Java implementation, lightweight and easy to integrate, modern Java/JS, V8 debug protocol support. In this case it will be a viable GraalJS alternative for us. As of now we switched off all the others, to minimise the support cost on our side.
Hope, this is somehow helpful for you.

[tuchida]

* Scriptable changes

This is quite a ways off, but if we implement class private members, is there going to be a breaking change required in Scriptable?

[p-bakker]

Good question and no idea.

Think this is one of the challenges we have if we want to bundle all breaking changes in one release: we need to access the impact of implementing of all missing EcmaScript features and see if they require breaking changes

Some of the Scriptable interface challenges I've run into so far:

• delete method having a void returnType (whereas in ES a delete operation must return boolean)
• disparity between the Abstract Object Method operations defined in ES and the Scriptable interface: the Scriptable interface doesn't align anymore with the operations ES defines on objects. Implementing Scriptable doesn't give you a Class that acts like a JavaScript object all the time. If you want that, you ought to extend NativeObject. When attempting to implement Proxy for example, this made things challenging: should Proxy support Scriptable or only classes that extend NativeObject?

And then there's the Callable interface which assumes the 'this' of a method call is a Scriptable, while that is an invalid assumption in strict mode, as primitive values as 'this' are allowed in strict mode and should not be boxes

[tuchida]

If we are implementing from scratch, wouldn't you also want to manage Property Descriptors with Scriptable?

[p-bakker]

If we are implementing from scratch, wouldn't you also want to manage Property Descriptors with Scriptable?

What about looking at all the Proxy traps and/or Reflect methods to see what a new/updated EcmaScript-compatible Scriptable interface should look like?

I think the Proxy/Reflect give us the definition of the 'contract' that 'objects' in EcmaScript should adhere to