Adds REST APIs for creating and provisioning a workflow

[joshpalis]

Description

Initial implementation, set up rest/transport actions, registration, pending state index implementations

This PR exposes two new APIs to CreateWorkflow (save) and to ProvisionWorkflow (execute). More details on the URI, payloads can be found on issue #49 .

Additionally, this PR integrates PR #65,

In this current state, the rest and transport actions to facilitate these requests have been set up. The CreateWorkflow API will be the main entry point in which the plugin's system indices are created (Global Context and State). Once created. the CreateWorkflow API will parse a use case template from the request content and either save to or update an existing entry within the Global Context index, returning the document ID as the workflow_id. Any insert into the Global Context entry will create a corresponding State index entry mapped with the workflow_id, defaulted to a NOT_STARTED state. Additionally, any update to a use case template will default its state back to NOT_STARTED.

The Provision API is responsible for executing the provisioning workflow for a use case template stored in the Global Context. The workflow_id is returned back to the user for use in querying provisioning status and the provision workflow is executed asynchronously on a separate thread, updating the state entry on success or failure.

Using test template (invalid due to self-connecting edge):

{
  "name": "semantic-search",
  "description": "My semantic search use case",
  "use_case": "SEMANTIC_SEARCH",
  "operations": [
    "PROVISION"
  ],
  "version": {
    "template": "1.0.0",
    "compatibility": [
      "2.9.0",
      "3.0.0"
    ]
  },
  "user_inputs": {
    "index_name": "my-knn-index",
    "index_settings": {}
  },
  "workflows": {
    "provision": {
      "nodes": [
        {
          "id": "create_index",
          "type": "create_index",
          "inputs": {
            "name": "user_inputs.index_name",
            "settings": "user_inputs.index_settings",
            "node_timeout": "10s"
          }
        }
      ],
      "edges": [
        {
          "source": "create_index",
          "dest": "create_index"
        }
      ]
    }
  }
}

Create API :

curl -XPOST "localhost:9200/_plugins/_flow_framework/workflow" -H "Content-Type:application/json" --data '{"name":"semantic-search","description":"My semantic search use case","use_case":"SEMANTIC_SEARCH","operations":["PROVISION"],"version":{"template":"1.0.0","compatibility":["2.9.0","3.0.0"]},"user_inputs":{"index_name":"my-knn-index","index_settings":{}},"workflows":{"provision":{"nodes":[{"id":"create_index","type":"create_index","inputs":{"name":"user_inputs.index_name","settings":"user_inputs.index_settings","node_timeout":"10s"}}],"edges":[{"source":"create_index","dest":"create_index"}]}}}'
{"workflow_id":"0IDAJIsBzSbcoaec4g95"}% 

OpenSearch logs successful Global Context Creation :

[2023-10-12T16:38:18,615][INFO ][o.o.n.Node               ] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] started
[2023-10-12T16:38:18,627][INFO ][o.o.g.GatewayService     ] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] recovered [0] indices into cluster_state
[2023-10-12T16:38:22,272][INFO ][o.o.p.PluginsService     ] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] PluginService:onIndexModule index:[.plugins-ai-global-context/EQigAvM-QsKlhEMMTfs0gw]
[2023-10-12T16:38:22,373][INFO ][o.o.c.m.MetadataCreateIndexService] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] [.plugins-ai-global-context] creating index, cause [api], templates [], shards [1]/[1]
[2023-10-12T16:38:22,379][INFO ][o.o.c.r.a.AllocationService] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] updating number_of_replicas to [0] for indices [.plugins-ai-global-context]
[2023-10-12T16:38:22,421][INFO ][o.o.p.PluginsService     ] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] PluginService:onIndexModule index:[.plugins-ai-global-context/EQigAvM-QsKlhEMMTfs0gw]
[2023-10-12T16:38:22,555][INFO ][o.o.c.r.a.AllocationService] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.plugins-ai-global-context][0]]]).
[2023-10-12T16:38:22,578][INFO ][o.o.f.w.CreateIndexStep  ] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] create index:.plugins-ai-global-context

Provision API :

curl -XPOST "localhost:9200/_plugins/_flow_framework/workflow/0IDAJIsBzSbcoaec4g95/_provision"
{"workflow_id":"0IDAJIsBzSbcoaec4g95"}

OpenSearch logs process sorting error due to self connecting edge :

[2023-10-12T16:38:37,617][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com] uncaught exception in thread [opensearch[dev-dsk-jpalis-2c-27c8aa11.us-west-2.amazon.com][opensearch_workflow_provision][T#1]]
java.lang.IllegalArgumentException: Edge connects node create_index to itself.
        at org.opensearch.flowframework.workflow.WorkflowProcessSorter.topologicalSort(WorkflowProcessSorter.java:110) ~[?:?]
        at org.opensearch.flowframework.workflow.WorkflowProcessSorter.sortProcessNodes(WorkflowProcessSorter.java:62) ~[?:?]
        at org.opensearch.flowframework.transport.ProvisionWorkflowTransportAction.executeWorkflow(ProvisionWorkflowTransportAction.java:135) ~[?:?]
        at org.opensearch.flowframework.transport.ProvisionWorkflowTransportAction.lambda$executeWorkflowAsync$4(ProvisionWorkflowTransportAction.java:122) ~[?:?]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:849) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]

Once the workflow steps are polished, we can proceed with full end to end testing of a valid use case template

Issues Resolved

#49
#61

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Merging #63 (8200441) into main (9b10b23) will decrease coverage by 0.07%.
The diff coverage is 81.51%.

@@             Coverage Diff              @@
##               main      #63      +/-   ##
============================================
- Coverage     81.74%   81.68%   -0.07%     
- Complexity      211      285      +74     
============================================
  Files            21       30       +9     
  Lines           838     1119     +281     
  Branches         96      125      +29     
============================================
+ Hits            685      914     +229     
- Misses          119      160      +41     
- Partials         34       45      +11     

Files	Coverage Δ
.../opensearch/flowframework/FlowFrameworkPlugin.java	100.00% <100.00%> (ø)
...g/opensearch/flowframework/common/CommonValue.java	66.66% <ø> (ø)
...lowframework/exception/FlowFrameworkException.java	40.00% <ø> (+40.00%)	⬆️
...arch/flowframework/indices/FlowFrameworkIndex.java	100.00% <ø> (ø)
...nsearch/flowframework/model/PipelineProcessor.java	100.00% <100.00%> (ø)
...g/opensearch/flowframework/model/WorkflowNode.java	90.54% <100.00%> (ø)
.../flowframework/transport/CreateWorkflowAction.java	100.00% <100.00%> (ø)
...ework/transport/CreateWorkflowTransportAction.java	100.00% <100.00%> (ø)
...owframework/transport/ProvisionWorkflowAction.java	100.00% <100.00%> (ø)
...earch/flowframework/transport/WorkflowRequest.java	100.00% <100.00%> (ø)
... and 8 more

[dbwiddis]

LGTM, would approve except with the TODOs not sure this is done.

[owaiskazi19]

Need to discuss more on the APIs. This would also require integ tests, let's create an issue for the same

[owaiskazi19]

Almost there. Looks good overall.

[owaiskazi19]

LGTM! Thanks for addressing the comments patiently

[dbwiddis]

LGTM, please create an issue for the TODO if there isn't one.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/opensearch-ai-flow-framework/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/opensearch-ai-flow-framework/backport-2.x
# Create a new branch
git switch --create backport/backport-63-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 014475d7e2968b723c916fb5c134d257f0b11578
# Push it to GitHub
git push --set-upstream origin backport/backport-63-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/opensearch-ai-flow-framework/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-63-to-2.x.

[joshpalis]

Seems the backport PR failed to get raised, @owaiskazi19 ill create a manual PR