[2.0.0] Discussion how to release 2.0.0 parser-js

[magicmatatjahu]

Of course, the release itself is not something difficult, but we need to discuss what implications the release of the new parser brings with it to the whole ecosystem of tooling, core and community ones.

Generator and other tools where we use ParserJS

First of all is integration problem in tooling. As ParserJS is treated as main tool for all other tools/application/generator templates that we have written in JS/TS, with 2.0.0 release we have problem how to make the transition from ModelAPI to IntentAPI very easy and "lazy" - that is, so that people don't have to quickly rewrite their code for the new API, and can do it optionally. Of course, after some time (a few months?) we should remove support for the old API and inform community about that, but we should have an "incubation" period for the new API.

This is difficult enough because we use the old Parser API in projects such as:

• Generator and all templates
• React component
• Studio to render left navigation and visualisation template
• Modelina to retrieve all schemas by AsyncAPIDocument.allSchemas() method (in old ModelAPI)
• Cupid (to retrieve servers and other things in old way)
• ServerAPI
• and few others...

Leaving the ModelAPI to be able to fix it in case of bugs

It's another problem related to the situation when we will release 2.0.0 of ParserJS, people will still use old ModelAPI but found some bugs (very small chance but there is) and would like to report it and fix it. We should support in first few months ModelAPI and IntentAPI in this same codebase (one solution) for such a situations.

Possible solution(s)

I have an idea to solve both problems, which seems to me the most cost-effective from the point of view of the work put in and the UX from the community side:

• we will move old ModelAPI to the current codebase in 2.0.0 version (to the next-major release), so we will have better validation (Spectral) but projects which use ModelAPI will be able to use the new Parser without any problems (and downgrades). Going with that solution we will introduce better maintainability of project, because we won't have to have separate branch only for 1.x.x version to problem described in Leaving the ModelAPI to be able to fix it in case of bugs section. Of course, we will provide the ability to use the Old or New api without big problems with migrations to have a good UX. We can have still two branches, for 2.x.x. and 1.x.x for some time, but it will only introduce problems with maintainability.
• in Generator we will introduce field apiVersion field to determine which version of Parser given template should use. By default it will be v1 (old API), but we will have also v2, so people will be able to lazy rewrite given template to the new API. There are also idea suggested by @fmvilas to determine version of ParserAPI by checking generator field. If template needs at least 2.0.0 version of Generator, then we will use IntentAPI, but if it points to the <2.0.0 version it will use ModelAPI. I don't like this approach, because then people will have to change Generator versions (install it globally with current Generator CLI) depending on the version of the ParserAPI that a given template uses. This will cause a lot of chaos in our community. Even if we made this integration by ourself with changing Generator versions in our tools themselves, it is still a bad solution because it leaves the problem of incompatibility of templates and will cause longer generation time. Also, another benefit with apiVersion field is that we don't need to release 2.0.0 of Generator, because everything will be handled in Generator code, so from Generator point of view, new ParserAPI will be non-breaking change.

Do you have ideas for other solutions, better ones? Take into account that the UX from the community side should be the best possible and that there should be as little effort as possible to switch to the new version of ParserAPI.

WDYT? cc @smoya @derberg @fmvilas @jonaslagoni

---

Progress on adapting code base to use new Parser-JS v2:

• Dependants of Parser-JS and Custom Schema Parsers:
  • https://github.com/asyncapi/cli - fix: upgrade to @asyncapi/parser v2 cli#547
  • https://github.com/asyncapi/generator - feat: upgrade @asyncapi/parser to v2.0.0 and use the new parser-api  generator#960
  • https://github.com/asyncapi/asyncapi-react -
  • https://github.com/asyncapi/server-api -
  • https://github.com/asyncapi/studio - fix: update parser-js to v2 studio#615
  • https://github.com/asyncapi/vs-asyncapi-preview -
• Templates

[fmvilas]

Yeah, forget about the idea we discussed. I support your idea of carrying the ModelAPI (I like the name 😄) forward into v2. In any case, let's make it clear and visible in the docs that the lifetime of such API is limited. And let's decide immediately how long we want to be supporting the old API. I suggest not longer than the end of January.

[magicmatatjahu]

@fmvilas Sure, thanks! This also needs to be discussed how much we want to have support for the old API. I was thinking until the release of 3.0.0 spec, but this time can be shortened, as you say to January or February, something like that.

[fmvilas]

Yeah, the reason I said January is because I'd love to get to 3.0.0 with only the intent-driven API. I know this is not going to happen for everyone but at least I think it should happen for our tools. That would be a great way to test the new API with a real use case and see if it really "survives" the new major version 😄

[smoya]

"End of January 2023" Seems what we are looking for then?

[derberg]

well, I like how optimistic you are with January but we cannot commit and deadlines now, really. API is not yet released, and we need some release candidates first to start playing with it, beta before the official release. So how can we say January 😄

if you want to pull in old API, and release both -> I'm interested in details on how you wanna do it 😄

I don't have a strong opinion, but IMHO we need no deadlines. We release betas, then just do 2.0 and people have as much time as they want to migrate, they just need to remember that we will not maintain 1.x and will not add new spec features there, that is it 🤷🏼 . So the only deadline we can really do is, that we will add new spec features to 1.x only once beta do not move to official 2.0

[magicmatatjahu]

@derberg

if you want to pull in old API, and release both -> I'm interested in details on how you wanna do it 😄

It will be very simple. We will create src/old-api folder in new project (next-major branch) and move all old models (JS classes) to new location and also from lib we will expose migrateToOldAPI function to convert new API to OLD (exactly retrieve parsed JSON from new AsyncAPiDocument class to the old one), so example of usage could look like:

import { Parser, migrateToOldAPI } from '@asyncapi/parser';

const parser = new Parser();
const parsed = parser.parse({ ... } // spec);
const oldAsyncAPIDocumentModel = migrateToOldAPI(parsed);

In generator we will have new field in template's config apiVersion and if someone will use v2 as value will use new API, but by default that field will point to the v1 (to make easy migration to new API in templates) and in Generator side we will pass to template new or old API :)

EDIT: of course after "incubation" period for new API, when we will remove old api we will release new major version of Generator to remove support for old api.

[derberg]

it sounds complicated 😄 why not just releasing new API only as 2.0 beta, then also generator 2.0 beta:

• and I can migrate my template by modifying https://github.com/asyncapi/nodejs-template/blob/master/package.json#L104 and that is it, why I need apiVersion?
• or I can just not migrate if I do not need to, because I already specify generator <2.0.0 ?

or am I missing something?

[magicmatatjahu]

@derberg With Fran we thought about such a solution with Generator release, but it has one problem from User experience perspective.

If we released a new version of Generator and version 2.x.x only will have support for ParserAPI version 2.x.x then we would have to have logic in CLI and other tools (that use Generator) that would switch Generator versions between 1.x.x and 2.x.x depending on the supported Generator version by the template. This is much more complicated to write and manage than an additional function that would convert the new API to the old one. So in my idea is to handle such a problems in one place not in N (where N is number of tools that we need to migrate). apiVersion is also good because if we will have another big breaking changes in ParserAPI (maybe in v4 AsyncAPI release), then we would not broke the entire tools ecosystem, but for some period of time we will have backward compatibility with old API.

[derberg]

from User experience perspective.

I do not think you look from user perspective but maintainer perspective 😉

From user's perspective, all is good. User does not have to do anything. We just release new versions that they can use or not.

only will have support for ParserAPI version -> from our perspective "only", as we as maintainers are aware of how our ecosystem of libraries is structured. But from a template developer, the change is gigantic, and it is really v2. We release beta, which they can use or not.

---

but yeah, while writing this comment, I realized that more complexity would end up in projects like CLI, Studio, and others. You are right 👍🏼

[magicmatatjahu]

PR for generator asyncapi/generator#833

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[smoya]

still relevant

[smoya]

New versions of the Custom Schema Parsers (Avro, Raml, and OpenAPI) are breaking compatibility so they can't be used without updating to Parser-JS v2.

The following repositories have custom schema parsers as dependencies.

• https://github.com/asyncapi/cli
• https://github.com/asyncapi/generator
• https://github.com/asyncapi/asyncapi-react
• https://github.com/asyncapi/server-api
• https://github.com/asyncapi/studio
• https://github.com/asyncapi/vs-asyncapi-preview

To migrate to the new Parser-JS V2, each repository must adapt the code to adopt Parser-JS v2 RC and use the RC of each Custom Schema Parser as well.
Once that happens, Parser-JS V2 should be ready to be released (only if all v3 BC features are already merged of course).

cc @derberg @fmvilas @magicmatatjahu

[smoya]

Regarding v3 features, all of them are missing.

The following is the list of features (I hope I don't miss any) from v3 that require changes in Parser-JS v2 and most probably Parser-API:

• feat: add new channels object spec#827 @fmvilas
• feat!: move operations to its own root object spec#806 @fmvilas
• feat: unify all referencing mechanisms spec#852 @fmvilas
• feat: replace server url with server host and pathname spec#888 @fmvilas
• feat: add more reusable objects to the components object spec#792 @magicmatatjahu
• feat: move root tags and externalDocs to the info object spec#794 @magicmatatjahu
• fix: remove action and channel fields from Operation Trait Object spec#860 @magicmatatjahu
• feat: add missed "metadata" fields in the Server/Channel/Operation Objects spec#796 @magicmatatjahu
• feat: request/response support spec#847 @GreenRover

Please reply in case you are happy to take this.

[magicmatatjahu]

@smoya We can merge this PR #708 and 90% of changes will be applied to the v3 models - then we can check what is missing and add it in another PRs.

[smoya]

Parser-JS codebase is up-to-date right now with all required changes for supporting AsyncAPI v2 atm.
There is only one PR in parser-API that should be merged so Parser-JS v2 can stick to that particular version of the API (mention in docs). But that's pretty minor and not so mandatory seems we can keep Parser-API been released as "alpha".

EDIT: PR merged

[smoya]

The release process has been started. Follow the progress in the following Slack thread: https://asyncapi.slack.com/archives/CQVJXFNQL/p1682327883697419

[smoya]

I updated the description of this issue to reflect the progress of adopting the new parser in the official tooling.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[jonaslagoni]

As v2 and v3 has already been released, I am closing this issue.