[incubator-kie-issues-834] Support Dynamic GraphQL query registering in Data Audit #1962

elguardian · 2024-01-19T10:47:10Z

…in Data Audit

data-audit/README.md

martinweiler · 2024-01-22T18:41:24Z

...-audit-quarkus/src/main/java/org/kie/kogito/app/audit/quarkus/GraphQLJPADataAuditRouter.java

+        }
+
+    }
+


I think what users might need here is an additional DELETE endpoint to clean up any queries that have been incorrectly registered or are no longer needed.

this will be addressed in another issue.

data-audit/data-audit-common/src/main/java/org/kie/kogito/app/audit/api/SubsystemConstants.java

...t/data-audit-common/src/main/java/org/kie/kogito/app/audit/graphql/GraphQLSchemaManager.java

...-audit-quarkus/src/main/java/org/kie/kogito/app/audit/quarkus/GraphQLJPADataAuditRouter.java

...data-audit-common/src/main/java/org/kie/kogito/app/audit/api/DataAuditStoreProxyService.java

.../src/main/resources/db/data-audit/postgresql/V1.1.0__Add Query audit dynamic registering.sql

kie-ci3 · 2024-01-24T15:52:28Z

PR job #4 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer

build-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-apps -u #1962 --skipParallelCheckout

NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/test/job/mcimbalek/job/kogito/job/main/job/pullrequest_jobs/job/kogito-apps-pr/job/PR-1962/4/display/redirect

Test results:

PASSED: 2857
FAILED: 1

Those are the test failures:

org.kie.kogito.app.audit.quarkus.QuarkusAuditQueryRegistryServiceTest.testRegisterQueryWithDateTime

Expected size: 1 but was: 6 in:
[{"eventDate"="2024-01-24T15:03:31.385Z", "eventId"="15ba01c1-2152-4699-9243-dd6c91315e0f"},
{"eventDate"="2024-01-24T15:03:32.275Z", "eventId"="47755a29-3373-4f61-a050-ff981c58145f"},
{"eventDate"="2024-01-24T15:03:31.438Z", "eventId"="7efb595a-58a4-4d5f-bd81-b0a1752d6733"},
{"eventDate"="2024-01-24T15:03:31.442Z", "eventId"="a81f8c5e-899a-47ce-bb4c-7bc439c48cae"},
{"eventDate"="2024-01-24T15:03:31.445Z", "eventId"="ae6f9ea7-0510-41a7-b099-ab3d8215b9a7"},
{"eventDate"="2024-01-24T15:03:31.457Z", "eventId"="283f92be-244f-41c0-b9af-25ec0d84b955"}]

kie-ci3 · 2024-01-24T15:54:16Z

PR job #5 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer

build-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-apps -u #1962 --skipParallelCheckout

NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-apps-pr/job/PR-1962/5/display/redirect

Test results:

PASSED: 2857
FAILED: 1

Those are the test failures:

org.kie.kogito.app.audit.quarkus.QuarkusAuditQueryRegistryServiceTest.testRegisterQueryWithDateTime

Expected size: 1 but was: 6 in:
[{"eventDate"="2024-01-24T15:00:53.568Z", "eventId"="80a1759f-eee3-43d2-86cd-0ec164ab8794"},
{"eventDate"="2024-01-24T15:00:54.458Z", "eventId"="dc68c298-3c16-4c85-9b2c-183150f30d0e"},
{"eventDate"="2024-01-24T15:00:53.625Z", "eventId"="6ed3bca5-2b31-4616-80fc-180806f53037"},
{"eventDate"="2024-01-24T15:00:53.629Z", "eventId"="b6aeb95a-0268-4253-81ad-69ba42969475"},
{"eventDate"="2024-01-24T15:00:53.632Z", "eventId"="7185e815-5f9d-4a2c-baa1-2d746d1cd36c"},
{"eventDate"="2024-01-24T15:00:53.644Z", "eventId"="e259817a-9852-4937-9933-ab8bdad22062"}]

...dons-data-audit-jpa-common/src/main/java/org/kie/kogito/app/audit/jpa/JPADataAuditStore.java

pefernan · 2024-01-24T16:12:43Z

@elguardian look at #1962 (comment) I'm also facing same issue locally

pefernan

Dropped a suggestion regarding a method name, the only issue I have is regarding the broken tests I mention on my previous comment.

Other than I'm ok with the PR as it is.

...t/data-audit-common/src/main/java/org/kie/kogito/app/audit/graphql/GraphQLSchemaManager.java

kie-ci3 · 2024-01-24T20:30:46Z

PR job #6 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer

build-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-apps -u #1962 --skipParallelCheckout

NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-apps-pr/job/PR-1962/6/display/redirect

Test results:

PASSED: 2854
FAILED: 3

Those are the test failures:

org.kie.kogito.index.ProcessDataIndexMongoDBIT.testProcessInstanceEvents

1 expectation failed.
JSON path errors doesn't match.
Expected: null
Actual: <[{message=Exception while fetching data (/UserTaskInstances[0]/schema) : Runtime service URL not defined, please review the kogito.service.url system property to point the public URL for this runtime., locations=[{line=1, column=85}], path=[UserTaskInstances, 0, schema], extensions={classification=DataFetchingException}}]>

org.kie.kogito.app.audit.quarkus.QuarkusAuditQueryRegistryServiceTest.testRegisterQueryWithDateTime

Expected size: 1 but was: 6 in:
[{"eventDate"="2024-01-24T18:35:37.901Z", "eventId"="2102ff47-7a30-4c6b-b78f-5ed718339f8e"},
{"eventDate"="2024-01-24T18:35:39.115Z", "eventId"="6f9bdacf-119a-4990-8d19-f082ab61e0f9"},
{"eventDate"="2024-01-24T18:35:37.975Z", "eventId"="f98f47bd-0669-46e3-be1f-13b7b11edc92"},
{"eventDate"="2024-01-24T18:35:37.980Z", "eventId"="e4557dda-6015-4504-99b6-e047f33dcc4e"},
{"eventDate"="2024-01-24T18:35:37.985Z", "eventId"="4b538011-5c45-42be-b11a-626fe3eade64"},
{"eventDate"="2024-01-24T18:35:37.999Z", "eventId"="d92de807-cc92-4645-8f11-334bb9a05c0c"}]

org.kie.kogito.trusty.service.common.KeycloakTrustyServiceIT.(?)

java.lang.RuntimeException: Failed to start quarkus

martinweiler · 2024-01-24T21:57:07Z

QuarkusAuditQueryRegistryServiceTest passes if it is run in isolation, but fails if run within the whole testsuite, so it seems the issue is related to data cleanup across tests.

I opened up apache/incubator-kie-issues/issues/857 to capture the need for a delete endpoint.

martinweiler · 2024-01-24T21:58:06Z

...ata-audit-quarkus/src/main/java/org/kie/kogito/app/audit/quarkus/GraphQLDataAuditRouter.java

+    }
+
+    @Route(path = DATA_AUDIT_QUERY_PATH, type = Route.HandlerType.BLOCKING, order = 2, methods = { GET })
+    public void blockingGraphQLHandlerGet(RoutingContext rc) {


@elguardian what is the purpose of the GET method for the query path?

we needed a way to verify that graphQL schema. If you add a new graphql query we needed a way to know the schema as it is build dynamically.

I still don't completely understand what this endpoint should return. It returns a 400 error in my tests due to a missing query.

Hi, probably you should update. It return a graphQL schema combining everything it jas within the jars plus the queries registered. try again.

Does the test I added below work for you?

that will fail because that is the query path and you need the query string. If you want to test this endpoint
?query=....

I will remove it to avoid confusion.

I think that having this GET to enable get the schema is a good idea. This will be helpful when we build tooling around the data-audit graphql interface.

IMO I wouldn't completely remove it, maybe we could enable it in dev mode.

My bad, I should have paid more attention. +1 to keep it in, but maybe with a test case or additional explanation in the README on how to use it.

...ata-audit-quarkus/src/main/java/org/kie/kogito/app/audit/quarkus/GraphQLDataAuditRouter.java

...ringboot/src/main/java/org/kie/kogito/app/audit/springboot/GraphQLAuditDataRouteMapping.java

...kus/src/test/java/org/kie/kogito/app/audit/quarkus/QuarkusAuditQueryRegistryServiceTest.java

martinweiler

Great work @elguardian

…in Data Audit (apache#1962) [incubator-kie-issues-834] Support Dynamic GraphQL query registering in Data Audit

[incubator-kie-issues-834] Support Dynamic GraphQL query registering …

1ca231a

…in Data Audit

elguardian mentioned this pull request Jan 19, 2024

Support Dynamic GraphQL query registering in Data Audit apache/incubator-kie-issues#834

Closed

elguardian added 4 commits January 19, 2024 11:50

add ddl for table

0e886ed

fix formatting

6aac347

add spring boot registering tests

0e62012

adding schema pretty print endpoint

4607620

elguardian requested review from martinweiler and pefernan January 22, 2024 09:32

martinweiler reviewed Jan 22, 2024

View reviewed changes

data-audit/README.md Outdated Show resolved Hide resolved

martinweiler reviewed Jan 22, 2024

View reviewed changes

pefernan requested changes Jan 23, 2024

View reviewed changes

elguardian added 2 commits January 24, 2024 10:39

revisit feedback

42516c8

formatting

5b921ba

elguardian requested review from martinweiler and pefernan January 24, 2024 09:44

elguardian added 2 commits January 24, 2024 14:29

feedback round #2

d2d1ea1

fix test base for failure registration

787eb31

pefernan reviewed Jan 24, 2024

View reviewed changes

...dons-data-audit-jpa-common/src/main/java/org/kie/kogito/app/audit/jpa/JPADataAuditStore.java Show resolved Hide resolved

pefernan requested changes Jan 24, 2024

View reviewed changes

...t/data-audit-common/src/main/java/org/kie/kogito/app/audit/graphql/GraphQLSchemaManager.java Outdated Show resolved Hide resolved

martinweiler reviewed Jan 24, 2024

View reviewed changes

fixing test a loading

5b516af

elguardian requested review from pefernan and martinweiler January 25, 2024 10:04

renamed method

3b07ede

pefernan requested changes Jan 29, 2024

View reviewed changes

...ata-audit-quarkus/src/main/java/org/kie/kogito/app/audit/quarkus/GraphQLDataAuditRouter.java Outdated Show resolved Hide resolved

...ringboot/src/main/java/org/kie/kogito/app/audit/springboot/GraphQLAuditDataRouteMapping.java Outdated Show resolved Hide resolved

fix graphql init

9e97e85

pefernan approved these changes Jan 29, 2024

View reviewed changes

martinweiler reviewed Jan 29, 2024

View reviewed changes

...kus/src/test/java/org/kie/kogito/app/audit/quarkus/QuarkusAuditQueryRegistryServiceTest.java Show resolved Hide resolved

remove get method in quarkus impl

a94dbc9

martinweiler approved these changes Jan 30, 2024

View reviewed changes

elguardian requested a review from martinweiler January 31, 2024 07:44

fix parenthesis

e198f8d

martinweiler approved these changes Jan 31, 2024

View reviewed changes

elguardian merged commit 5bd2b09 into apache:main Feb 1, 2024
5 checks passed

elguardian deleted the incubator-kie-issues-834 branch February 1, 2024 08:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[incubator-kie-issues-834] Support Dynamic GraphQL query registering in Data Audit #1962

[incubator-kie-issues-834] Support Dynamic GraphQL query registering in Data Audit #1962

elguardian commented Jan 19, 2024 •

edited by yesamer

Loading

martinweiler Jan 22, 2024

elguardian Jan 23, 2024

kie-ci3 commented Jan 24, 2024

kie-ci3 commented Jan 24, 2024

pefernan commented Jan 24, 2024

pefernan left a comment

kie-ci3 commented Jan 24, 2024

martinweiler commented Jan 24, 2024

martinweiler Jan 24, 2024

elguardian Jan 25, 2024

martinweiler Jan 25, 2024

elguardian Jan 29, 2024

martinweiler Jan 29, 2024

elguardian Jan 30, 2024

elguardian Jan 30, 2024

pefernan Jan 30, 2024

martinweiler Jan 30, 2024

martinweiler left a comment

[incubator-kie-issues-834] Support Dynamic GraphQL query registering in Data Audit #1962

[incubator-kie-issues-834] Support Dynamic GraphQL query registering in Data Audit #1962

Conversation

elguardian commented Jan 19, 2024 • edited by yesamer Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kie-ci3 commented Jan 24, 2024

kie-ci3 commented Jan 24, 2024

pefernan commented Jan 24, 2024

pefernan left a comment

Choose a reason for hiding this comment

kie-ci3 commented Jan 24, 2024

martinweiler commented Jan 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martinweiler left a comment

Choose a reason for hiding this comment

elguardian commented Jan 19, 2024 •

edited by yesamer

Loading