[FEATURE] Separate Local Model Registration to support Custom, Pretrained and Sparse Encoding Models

[joshpalis]

Is your feature request related to a problem?

Coming from flaky integration test failures due to the memory limitations of Github action runners which facilitate integration tests (Reference), it has been determined that mitigation would require using a smaller model to test out local model registration within an integration test cluster.

The amazon/neural-sparse/opensearch-neural-sparse-tokenizer-v1 has been determined to be the smallest model we can use for testing (Documentation) , however the required fields for registering a sparse encoding model is different than the required fields for registering a custom text embedding model.

In order to replace the model used in integration testing with this sparse encoding model, it is required to separate out the RegisterLocalModelStep into a RegisterCustomLocalModelStep and RegisterSparseEncodingLocalModelStep. Additionally, we require support for registering an OpenSearch -provided pretained model, which does not require a URL.

What solution would you like?

The RegisterCustomLocalModelStep relates to the following documentation and will have the following required and optional fields :

Required keys :

• name
• version
• model_format
• function_name
• model_content_hash_value
• url
• model_type
• embedding_dimension
• framework_type

Optional keys :

• description
• model_group_id
• all_config
• deploy

The RegisterSparseEncodingModelStep relates to the following documentation and will have the following required and optional fields

Required keys :

• name
• version
• model_format
• function_name
• model_content_hash_value
• url

Optional keys :

• description
• model_group_id
• deploy

The RegisterLocalPretrainedModelStep relates to the following documentation and will have the following required and optional fields

Required keys :

• name
• version
• model_format

Optional keys :

• description
• model_group_id
• deploy

[dbwiddis]

Suggestion:

• rename existing RegisterLocalModelStep into an abstract parent step.
• add an abstract method to get the required and optional keys
• create 3 subclasses implementing those required/optional keys methods, each with their own NAME