MVP integration of extensions catalog with AsyncAPI tools to make extension catalog useful

[derberg]

Reason/Context

I don't think there were many situations in the past when someone asked about an option to have a kind of official extension. This feature is still important though. People use extensions and we should support at least the official ones in our tools. There is a lot of work that has to be done and IMHO we need to get work done in kind of stages.

MVP is where I'd like us to focus on alpha implementation (stage 1), simple solution that will serve us (spec maintainers) and community (folks that want to add new features to the spec) as a place where the possible features for spec can be first implemented as extensions.

So RFC like asyncapi/spec#584 can be battle-tested much faster as an extension first to speed up work on adding the feature to the spec later with more confidence.

Description

What needs to be done in short?

1. I want to be able to create the following doc with x-twitter extension

asyncapi: '2.3.0'
info:
  title: Account Service
  version: 1.0.0
  description: This service is in charge of processing user signups
  x-twitter: AsyncAPISpec
channels:
  user/signedup:
    subscribe:
      message:
        $ref: '#/components/messages/UserSignedUp'
components:
  messages:
    UserSignedUp:
      payload:
        type: object
        properties:
          displayName:
            type: string
            description: Name of the user
          email:
            type: string
            format: email
            description: Email of the user

2. I want AsyncAPI JavaScript parser to provide validation error, after validating x-twitter extension against https://github.com/asyncapi/extensions-catalog/blob/master/extensions/twitter/0.1.0.yaml. The error should tell me that my twitter handle doesn't respect a given pattern with @ at the beginning of twitter handle. I want react component to throw this validation error when I check this document in Studio
3. When I fix my document based on the above validation errors, I see react component renders information about twitter handle. MVP should not cover concept of react component plugins for that extension, so the component renders also twitter icon and uses provided twitter handle in link to twitter profile.
4. Using already provided helpers https://github.com/asyncapi/parser-js/blob/master/lib/mixins/specification-extensions.js should be enough for now. If we have time, we can think about extension plugins
5. Extensions should be documented in extensions repo, and then this document should be also available like the spec, in the website. No need for some fancy UI for now, can be done like in case of the spec, markdown file that lists all extensions and describes them.

---

Requirements

JSON Schema - Validation

To enable validation in the parser, we need JSON Schema for AsyncAPI to contain JSON Schemas of individual extensions. Just like it will be with bindings -> https://github.com/asyncapi/spec-json-schemas/tree/next-major-spec/bindings

This means that contribution flow for extensions just like for spec and bindings will require a PR not only with extension definition to extensions-catalog but also with the schema to spec-json-schemas repo

TODO -> we need a GitHub issue in spec-json-schemas with info that this will be done, and how -> resulting in a PR to introduce first extension

asyncapi/spec-json-schemas#280

Simple governance

Since the goal is to get the first extensions officially published and promoted, we need a super simple guide on how people can contribute more extensions.

Current CONTRIBUTION.md is generic. We need to get rid of it.

This is a guide for the spec. But I would not be too formal and just duplicate what we have in spec. Extensions are something different and their goal is to introduce possible features quicker, to explore their adoption. This is why the contribution process should not be that formal, with so many stages.

I want to contribute this extension -> #11. This is a good example that could serve as an example for contribution guide -> how can I contribute it?

TODO ->

• we need a PR that will add extensions repo to ignore list here in .github repo
• we need fresh contribution guide. So open up an issue in this repo and propose something.

Rendering in UI

Extensions are part of the spec. This is why IMHO we do not need any specific plugins/rules and stuff like that for tooling to support extensions. This is an individual decision of every tooling maintainer on how they want to support extensions.

We just need to pick the first extension, the simplest one, which we already have 💪🏼 -> twitter. End enable it in tools where it is needed, which in this case it is simple -> we need it only for AsyncAPI-documentation-related use cases.

TODO -> open up an issue in react-component to discuss how and where in the code would be the best way to provide extensions-related components. And then provide one for x-twitter extension.

Parsing extensions

At the moment, we do not have any mechanism for dedicated parsing of extensions or bindings. Official JS Parser is able just to return the value of extension, but that is it.

We need an issue in the parser-js repo to discuss the best approach. If we really need to have a separate plugin for extension parsing, or can extension-related models be added directly to parser code.

TODO -> open up an issue in parser-js repo to discuss parsing of extensions. Twitter extension is not a good example as it is just a string. We can base discussion on #11. This extension is more complex, as it is a list of titles and contents. I would like parser to provide me some good helpers that will make it easier for me to access the contents of this extension, like getAllTitles.

Extensions list in website

We need a place in website -> https://www.asyncapi.com/docs. Probably under Specification section.

There has to be a short explanation of what is the purpose of extensions, and then display list of all of them.

I don't think we need any fancy UI with filters etc. Not for MVP. We will have just a few extensions at the beginning, so let us see how this will work after all. When more and more will come, we can always think of some nice UI 🤷🏼

TODO ->

• create issue in this repo, that will be scoped around building automation for pushing extensions definitions from this repo, to website, whenever some change is introduced. We need typical release pipeline + automation like we have with spec when on release we push it out to the website

[hkaur008]

Will go through the idea! and solution

[Sihamtahi]

Hello,
I want to work on this project because during my former internship, we needed an extension to document the work but it was not possible at that time because I did not master enough the extensions (time constraint too). So, in this case I could carry out the need described in this issue and propose other possible extensions for later (in addition to those of the catalog).

[derberg]

@Sihamtahi this idea was selected for AsyncAPI Mentorship 2022 -> asyncapi/community#376 (comment)

You are the only one that volunteered to work on this one + you also contributed to the project by helping @Barbanio a lot with feedback on the official AsyncAPI training - would you like to become an AsyncAPI-sponsored mentee for this idea? 🙏🏼 I would be your mentor for the coming months

[Sihamtahi]

Hello @derberg, thank you for this great news 🙏🏼. Happy to be an AsyncAPI-sponsored mentee 🎉🎉🎉

[derberg]

@Sihamtahi I finally found time to write down requirements 😄 I'll explain them all on our call

I will also on my own create a separate issue in this repo, to discuss its cleanup, as it is not a bit too complex

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[derberg]

change of scope:

• we have initial stuff in extension-catalog
• also initial stuff in schema done feat: add extensions support spec-json-schemas#506

@sambhavgupta0705 react is out of scope, so close your PR - but leave issue open, just write you will not continue working on it.

so instead of react, there is much more smaller thing to do - in short: test if what we did makes sense

• so now add a small PR to extension-catalog that introduces a new extension -> for linkedin
• using https://github.com/asyncapi/spec-json-schemas?tab=readme-ov-file#extensions instruction open PR to add linkedin extension schema

this will validate the work is completed and you are good to go

questions?

[sambhavgupta0705]

@derberg

so now add a small PR to extension-catalog that introduces a new extension -> for linkedin

for this I need to just replicate the x.md file and make the adjustements?

using https://github.com/asyncapi/spec-json-schemas?tab=readme-ov-file#extensions instruction open PR to add linkedin extension schema

OKAY

[derberg]

@sambhavgupta0705 congrats on completing the mentorship

the minimum is here:

• catalog refreshed
• automation that publish new extensions works: https://github.com/orgs/asyncapi/teams/conference_coordination_wg/members
• integration with json schema is here https://github.com/asyncapi/spec-json-schemas?tab=readme-ov-file#extensions
• whole discussion about react component that you run for months resulted in SAP opening a PR to support extensions in the component feat: custom extension rendering asyncapi-react#994

MVP fulfilled, please reach out to @AceTheCreator for details how to close mentorship and apply for payment.
And make some noice on socials!!! 😄

[sambhavgupta0705]

Thanks @derberg for always helping me out🥺😁