[MD] Support connection to AWS OpenSearch domain with user based Sigv4

[zhongnansu]

Task Breakdown

• user based Sigv4 to connect to Amazon OpenSearch domain

  • Frontend:

    • Create page add new auth type as a radio button for user to select
    • input boxes for "AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, region", mask the keys
      • input validation
    • helper text if needed
    • Disable edit auth content or type for Sigv4, due to the limitation in next section.
    • integrate with test connection on client side.

  • Backend:

    • create new auth type and data model
    • encrypt user input fields
    • decrypt and establish connection
    • support legacy client (elasticsearch-js client)
    • error handling (see if we need extra)
    • figure our client spawning strategy.
    • Integrate with the recent test connection feature

Limitation

• Due to fact that, opensearch js client doesn't allow spawning child with different Connection class(in our case the class is SigV4AWSConnection). I plan to disable sigv4 credential update for existing datasource in the 1st iteration. So that we can still benefit from client pooling.

Need UX/UI input

• what should be good names for all the new user inputs that comes with this new auth type?
  • 1. auth type name. E.g. "AWS" or "SigV4", etc
  • 2. AWS_ACCESS_KEY_ID
  • 3. AWS_SECRET_ACCESS_KEY
  • 4. region
• My plan is to follow the existing UI/UX paradigm of basic auth to implement AWS auth type, in MD. I'll add additional auth type as a new radio button for user to select. All user inputs will be masked on UI, and encrypted in the backend. To update the auth info, user will need to re-enter all credentials. Are you good with this @KrooshalUX
• ref

[kavilla]

Hey @zhongnansu is there a way to make this configurable in a way that treats this as an integration. So that it easy for future providers add their own integration.

Is this safe to do?

Also, as a cluster admin or someone with higher permissions I think I would like the ability to even filter the list that in this drop down. If this lists get large or if I have any legal worries about referencing/using 3rd party services then I can prevent my users from seeing specific auth methods (ie AWS).

[KrooshalUX]

@zhongnansu I think since we are now adding more authentication types, we should re-visit the design using EuiSuperSelect or EuiComboBox limited to a single selection, as it is a searchable interface which when considering builders providing added integrations, this list may extend 5-7 choices (my max recommendation for SuperSelect. Additionally, should we consider adding calrifying detail to Username & Password, that it is intended for OpenSearch clusters (unless I am mistaken).

We should set a default for this selection, and I am wondering what your thoughts are here @zhongnansu ?

@kavilla Great thoughts here - I think the administration end of this is out of scope for 2.5, but I think we need to consider it as part of the roadmap. I believe I will need to work with @kamingleung on administration settings.

[zhongnansu]

@KrooshalUX Currently, we don't have future plans for more auth types other than no_auth, basic_auth, sigV4. The SuperSelect will be a good option when we decide we'll support more. Guess 3 options for radio button is acceptable? Considering if we want to catch the next release is early Jan, I'd say we choose implement with radio options(less effort), if you don't have strong opinions.

Not only Usename & Password, all auth types are intended only for OpenSeach clusters. Currentlu We support 1 datasource type (OpenSearch), but multiple auth types.

Default selection auth type is currently No auth

[zhongnansu]

Existing client management in data source module doesn't fully work with spawning AWS opensearch client. Researching on additional cache strategy for aws clients only

[joshuarrrr]

Reopening to track this for a release - the PR merged to main, but we need a further solution with the client library to backport to 2.x.

[joshuarrrr]

closed via #3058 and #3477