Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: jq rules engine refactor/tests #589

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Conversation

brendanobra
Copy link
Contributor

@brendanobra brendanobra commented Aug 5, 2024

What

  1. ripple core/main as a library (ostensibly to enabled automated testing of jq rules)
  2. Rules engine refactor to a) decouple rule processing workflow from tokio b) make smaller things that are more easily reasoned about, are easier to unit test
  3. Explicit/deterministic error reporting/handling for jq rules
  4. Unit test related changes a) defaults for many oft used classes b) unit tests for broker + rules
  5. Added Default implementation (both explicit and derived , where possible)

Why

Automated testing of jq rules is currently not possible
Errors are obfuscated.

How

Refactor to expose appropriate components, more idiomatic use of rust' type system

Test

cargo test

Checklist

  • [ X] I have self-reviewed this PR
  • [ X] I have added tests that prove the feature works or the fix is effective

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link

github-actions bot commented Aug 7, 2024

Code Coverage

Package Line Rate Health
core.main.src.state.cap 45%
core.sdk.src.extn.client 91%
core.sdk.src.api.observability 68%
core.sdk.src.api.gateway 85%
core.main.src.service.apps 47%
core.sdk.src.utils 53%
device.thunder_ripple_sdk.src.processors 10%
core.main.src.processor 0%
core.launcher.src.manager 7%
core.main.src 0%
core.sdk.src.api.device 77%
core.main.src.service 32%
core.sdk.src.framework 75%
core.main.src.bootstrap 0%
core.main.src.firebolt 4%
core.main.src.bootstrap.extn 0%
openrpc_validator.src 90%
device.thunder_ripple_sdk.src.events 4%
core.sdk.src.extn.ffi 80%
core.tdk.src.gateway 100%
core.sdk.src.api 78%
core.sdk.src.extn 83%
device.thunder.src 0%
core.main.src.broker 45%
core.main.src.broker.thunder 54%
device.thunder.src.bootstrap 0%
distributor.general.src 2%
core.main.src.bootstrap.manifest 0%
core.tdk.src.utils 0%
core.main.src.state 26%
core.sdk.src.api.manifest 91%
core.launcher.src 0%
core.main.src.service.extn 43%
core.main.src.processor.storage 0%
core.main.src.utils 29%
device.thunder_ripple_sdk.src.bootstrap 0%
device.thunder_ripple_sdk.src.processors.events 0%
core.sdk.src.api.firebolt 78%
core.main.src.firebolt.handlers 8%
device.thunder_ripple_sdk.src 15%
device.mock_device.src 52%
device.thunder_ripple_sdk.src.client 74%
core.sdk.src.api.distributor 84%
Summary 44% (17923 / 40950)

Minimum allowed line rate is 42%

pub fn get_event_id_from_method(method: Option<String>) -> Option<u64> {
method.and_then(|m| {
let event: Vec<&str> = m.split('.').collect();
event.first().and_then(|v| v.parse::<u64>().ok())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some safety for bad methods

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noted, will map/? it

@@ -502,7 +540,7 @@ pub trait EndpointBroker {
}

/// Generic method which takes the given parameters from RPC request and adds rules using rule engine
fn apply_request_rule(rpc_request: &BrokerRequest) -> Result<Value, RippleError> {
fn apply_request_rule(rpc_request: &BrokerRequest) -> Result<Value, JqError> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep them as Ripple Error instead of JqError

Ripple error is already an enum which does the same for Deny Reason

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RippleError is overloaded imho, which is why i leaned into this more rust idiomatic apporach.

pub fn run_broker_workflow(
broker_output: &BrokerOutput,
broker_request: &BrokerRequest,
) -> Result<BrokerWorkflowSuccess, BrokerWorkFlowError> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here lets keep RippleError instead of BrokerWorkFlowError and update the Enum

Copy link
Contributor Author

@brendanobra brendanobra Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, the goal of this approach is to provide more information, with a better chance of figuring out where the error was generated. What is the objective for using RippleError?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants