-
Notifications
You must be signed in to change notification settings - Fork 212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fix #3721] Optimize event grouping #3739
Conversation
f2b8280
to
65e8d11
Compare
PR job Reproducerbuild-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-runtimes -u #3739 --skipParallelCheckout NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-runtimes-pr/job/PR-3739/18/display/redirect Test results:
Those are the test failures: |
4f828a8
to
c15c0fd
Compare
PR job Reproducerbuild-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-runtimes -u #3739 --skipParallelCheckout NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-runtimes-pr/job/PR-3739/21/display/redirect Test results:
Those are the test failures: PR check / Build projects / org.kie.kogito.it.JDBCOptimisticLockingIT.testPersistencejava.util.concurrent.CompletionException: java.lang.RuntimeException: Unable to start Quarkus test resource class org.kie.kogito.testcontainers.quarkus.PostgreSqlQuarkusTestResourcePR check / Build projects / org.kie.kogito.it.JDBCPersistenceIT.testPersistencejava.util.concurrent.CompletionException: java.lang.RuntimeException: Unable to start Quarkus test resource class org.kie.kogito.testcontainers.quarkus.PostgreSqlQuarkusTestResource |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, outstanding work @fjtirado !
api/kogito-events-core/src/test/java/org/kie/kogito/event/process/ProcessEventsTest.java
Show resolved
Hide resolved
api/kogito-events-core/src/test/java/org/kie/kogito/event/process/ProcessEventsTest.java
Show resolved
Hide resolved
7211e81
to
a863046
Compare
LOGGER.debug("Deserialize user task instance data event: {}", node); | ||
String type = node.get("type").asText(); | ||
|
||
switch (type) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit; maybe in an upcoming change, but in ProcessInstanceDataEvent
, you created enum types for it.
PR job Reproducerbuild-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-runtimes -u #3739 --skipParallelCheckout NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-runtimes-pr/job/PR-3739/27/display/redirect Test results:
Those are the test failures: |
* [Fix apache#3721] Optimize event grouping * [Fix apache#3721] Adding addons * [Fix apache#3721] Refactoring KogitoIndexConverter
Fix #3721
Merge with apache/incubator-kie-kogito-apps#2122
When property
kogito.events.grouping.binary
is set, process instance events collection is sent in binary format, which takes less space than sending the collection in json.The binary format is propietary, but it does not include any java stuff (besides the fact that is using Java byte stucture approach), so it is theoretically possible to interpret it from a different language.
The binary format reduce space by taking advantage of the following facts:
If after passing to binary, we still need to reduce the event size further,
kogito.events.grouping.compress
property can be set to true. This will gzip the resulting byte[] at the cost of some performance.