Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[incubator-kie-issues-974] Document data index in compact architecture #2002

Merged
merged 2 commits into from
Mar 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 91 additions & 3 deletions data-index/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,92 @@
## Kogito Data Index Service
# Kogito Data Index Service

Data Index is a subsystem/component responsible to store a snapshot of the last state of the process instance state.

The system supports two different types of deployment:
* Compact Architecture: as component deployed within the application
* Distributed Architecture: as component deployed independently as service

Due to the nature of the system it support different types of storage

* postgresql
* infinispan
* mongodb
* jpa

At present for compact configuration it only support quarkus runtime. For distributed supports both runtimes quarkus and springboot.


## Compact architecture

The next dependency is added for quarkus and being able to use in-vm transport tier.
The same dependency is used to specify the storage of the data index and the transport tier at the same time.


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two sentences starting with "The next dependency" could be a bit misleading. Maybe start the second one with "The same dependency" (and then remove "at the same time" at the end).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


<dependency>
<groupId>org.kie</groupId>
<artifactId>kogito-addons-quarkus-data-index-persistence-postgresql</artifactId>
</dependency>



As this dependency will include the storage as well and the in-vm transport.



<dependency>
<groupId>org.kie</groupId>
<artifactId>kogito-addons-quarkus-data-index-persistence-<storage></artifactId>
</dependency>


where storage can be:
* infinispan
* mongodb
* postgresql

Current system

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would help to add a sentence to describe the diagram, eg:

## How data index works

![data-index](docs/data-index.png "Data Index")

The concepts to understand the above picture are:
*   Transport: the medium used to transfer events between the runtime and the data index service. The transport is http in distributed environments, or in-vm for compact architecture.
*   Storage: the persistence tier of the data index component.
*   Data index: the main component responsible for creating/updating the data index, and for providing query capabilities.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

![data-index](docs/data-index.png "Data Index")


The concepts to understand the above picture are:
* Transport: the medium used to transfer events between the runtime and the data index service. The transport is http in distributed environments, or in-vm for compact architecture.
* Storage: the persistence tier of the data index component.
* Data index: the main component responsible for creating/updating the data index, and for providing query capabilities.


## Querying

Data index support queries through graphQL (basic.schema.graphqls). For using the endpoint you just need to explore in your deployment the URI:

http://localhost:8080/<root path>/graphql-ui/

to start working with the queries defined by the schema aforementioned.

## Distributed deployment

For distributed you need to specify and addons you will need to deploy the data index as service and include the event publisher



<dependency>
<groupId>org.kie.kogito</groupId>
<artifactId>kogito-addons-springboot-events-process-kafka</artifactId>
</dependency>



or for quarkus


<dependency>
<groupId>org.kie.kogito</groupId>
<artifactId>kogito-addons-quarkus-events-process-kafka</artifactId>
</dependency>



More information about the Data Index images in here:
[https://github.com/apache/incubator-kie-kogito-images?tab=readme-ov-file#kogito-data-index-component-images](https://github.com/apache/incubator-kie-kogito-images?tab=readme-ov-file#kogito-data-index-component-images)

Documentation:
https://github.com/kiegroup/kogito-runtimes/wiki/Data-Index-Service
Binary file added data-index/docs/data-index.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading