Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #97

Open
wants to merge 140 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
140 commits
Select commit Hold shift + click to select a range
fa5167c
expr based condition
balamg Oct 3, 2018
c4a62bf
expr based condition
balamg Oct 20, 2018
ac8f424
Merge branch 'master' into expr
balamg Oct 20, 2018
be0d36d
expr based condition
balamg Oct 21, 2018
1a28a29
expr based condition
balamg Oct 23, 2018
98283b6
expr based condition
balamg Oct 23, 2018
cb3df5d
Expression syntax for conditions
balamg Mar 4, 2019
b8a741d
Refactored to use the new expression API of project-flogo/core
balamg Mar 5, 2019
2b51fea
iterator interface for join table iteration
balamg Dec 6, 2018
556f7a2
assign integer id to jointablerows
balamg Dec 6, 2018
6ec22a2
handle holds ids of tables and rows
balamg Dec 6, 2018
59c5dd2
refactor for jointables and rows
balamg Dec 8, 2018
1d11109
refactor for jointables and rows
balamg Dec 9, 2018
69c5fc8
refactor for jointables and rows
balamg Dec 9, 2018
dca233e
refactor for jointables and rows
balamg Dec 10, 2018
1b9bf49
refactor for jointables and rows
balamg Dec 11, 2018
2ec7942
refactor interfaces in order to separate out and organize in-mem impl…
balamg Dec 12, 2018
e9a9738
refactor interfaces in order to separate out and organize in-mem impl…
balamg Dec 13, 2018
61efc06
configuration based impl
balamg Dec 19, 2018
448b0b6
need the condition name to get correponding join tables
balamg Dec 20, 2018
204eec4
refactor
balamg Dec 23, 2018
253adbe
refactor
balamg Dec 23, 2018
bc213e8
refactor
balamg Dec 23, 2018
8c970dc
refactor
balamg Dec 25, 2018
2dce1eb
refactor
balamg Dec 25, 2018
9f2b44d
refactor
balamg Dec 25, 2018
4e5097a
refactor
balamg Dec 25, 2018
35a141d
rename and refactor
balamg Dec 25, 2018
fa6c281
added redis utility methods
balamg Dec 28, 2018
c686129
redis util methods
balamg Dec 30, 2018
61297b6
misc bug fixes
balamg Jan 2, 2019
88f334f
idgen fixes
balamg Jan 2, 2019
607fa85
refactored interfaces and register a post-rtc in main
balamg Jan 2, 2019
1c8d3f0
working redis store impl and some bug fixes related to context
balamg Jan 2, 2019
543f4e2
redis store impl for join tables - wip
balamg Jan 3, 2019
20a9732
wip redis impl for jointables
balamg Jan 3, 2019
21816ab
wip redis impl for jointables
balamg Jan 3, 2019
4971fac
wip redis impl for jointables
balamg Jan 3, 2019
4db3d55
wip redis impl for jointables
balamg Jan 3, 2019
1e034b3
wip mem impl for hdlrefs
balamg Jan 4, 2019
7d523f1
wip redis impl for hdlrefs
balamg Jan 4, 2019
02c5913
wip redis impl for hdlrefs
balamg Jan 4, 2019
f833416
wip redis impl for hdlrefs
balamg Jan 5, 2019
3ab94e9
wip redis impl for hdlrefs
balamg Jan 5, 2019
6e0f10a
rule delete impl
balamg Jan 7, 2019
acb964f
refactor and cleanup
balamg Jan 7, 2019
241fe37
use in-memory implementation by default
balamg Jan 7, 2019
098bed9
merged master (project-flogo refactor and later changes)
balamg May 2, 2019
b0f3f3d
fixed configuration issues
Jun 11, 2019
9d68b5c
fix default rs config
Jun 11, 2019
26815d2
Merge branch 'master' into exprnew
Jun 28, 2019
6af83a6
fixed compilation errors in tests
Jul 4, 2019
fa00baa
Merge branch 'master' into exprnew
Jul 31, 2019
b45a266
fix go mod
Jul 31, 2019
f66fa96
flogo/flogo.json support for expression based conditions
Aug 2, 2019
8c80150
add go tests for examples
LakshmiMekala Aug 9, 2019
a636339
update kafka go test to kill running docker containers
LakshmiMekala Aug 9, 2019
eea8195
incorporating PR review comments
LakshmiMekala Aug 12, 2019
dda01ea
changing fn name captureout to capturestdout
LakshmiMekala Aug 13, 2019
33fa36d
assigned tuple store to rete & support for redis to rete
nareshkumarthota Aug 19, 2019
64a8568
Merge branch feature-invoke-activity
rameshpolishetti Aug 20, 2019
398290a
fixing rsconfig for flogo action model
nareshkumarthota Aug 21, 2019
9eea19e
remove unused sanity sh files
LakshmiMekala Aug 26, 2019
051b0b5
Intial version of decision table activity
rameshpolishetti Aug 26, 2019
c208501
handling null pointer for condivar in joinnode
nareshkumarthota Aug 27, 2019
d11ad33
Update app name in flogo.json
rameshpolishetti Aug 27, 2019
6c16b51
reverting conditionvar nil check change
nareshkumarthota Aug 28, 2019
3530432
decision table information fields added
nareshkumarthota Aug 30, 2019
62dc8e0
added decision table readme
nareshkumarthota Sep 5, 2019
beee2c1
Added ClearSessions function
pointlander Sep 12, 2019
cb2a6af
Fixed race condition
pointlander Sep 13, 2019
ae2e3c1
Added GetPathForResource function
pointlander Sep 13, 2019
2ada749
Fixed races in ruleapi package
pointlander Sep 13, 2019
f045de1
Added race condition test
pointlander Sep 16, 2019
aad89ad
Added memory leak test
pointlander Sep 16, 2019
8894b65
Added same tuple memory leak mode
pointlander Sep 16, 2019
2f39c28
improvised example for decision table
nareshkumarthota Sep 18, 2019
a646bdf
Merge branch 'feature-example-gotests' into develop-tests
pointlander Sep 18, 2019
b6c8052
Fixed race condition in test
pointlander Sep 18, 2019
8371efa
Merge pull request #76 from project-flogo/develop-tests
pointlander Sep 18, 2019
b1fd932
resolved conflicts from decision table branch and fixed go tests
nareshkumarthota Sep 19, 2019
15da903
Updated go.sum
pointlander Sep 23, 2019
776aca7
credit card example with decision table is added
nareshkumarthota Sep 24, 2019
3f94ff8
updated readme
nareshkumarthota Sep 24, 2019
7d14a47
Merge branch 'develop' into develop-store
pointlander Sep 27, 2019
dadd27b
Added status to handle
pointlander Sep 30, 2019
b002665
Refactor join node name code
pointlander Oct 1, 2019
754f1b7
Support parallel Assert and Delete of different tuples
pointlander Oct 2, 2019
b1a67ba
Added mutexes to memory store
pointlander Oct 3, 2019
7154860
Garbage collection
pointlander Oct 3, 2019
5bccdba
Redis store tests
pointlander Oct 8, 2019
2622077
Check for errors in tests
pointlander Oct 8, 2019
e84cb44
Add tuple to store when added to join table
pointlander Oct 9, 2019
f0dcce0
Improved handles
pointlander Oct 12, 2019
60cd8c0
Added mutex to redis JtService
pointlander Oct 14, 2019
a74526d
Refactor configuration
pointlander Oct 14, 2019
3149f9c
Get store config from environment
pointlander Oct 14, 2019
508ab60
Added dtable tests
pointlander Oct 16, 2019
110bffe
Updated dtable readme for redis store support
pointlander Oct 16, 2019
7419bb4
Removed print
pointlander Oct 17, 2019
b403946
Removed global redis handle
pointlander Oct 17, 2019
1e93ad9
Improved support for ttl of 0
pointlander Oct 18, 2019
57c9948
Merge pull request #79 from project-flogo/develop-store
pointlander Oct 18, 2019
9baaf04
Use different port for redis test
pointlander Oct 21, 2019
220ee4d
Use different redis port for ruleapp test
pointlander Oct 21, 2019
69f6482
Improved handle logic
pointlander Oct 24, 2019
b17aaa1
Allow the user to select consistency or performance
pointlander Oct 25, 2019
af2b2f1
Add excel file support for decision table
rameshpolishetti Oct 26, 2019
81e3bc0
Support for keydb
pointlander Oct 29, 2019
3117b11
excel file added for creditcard dt example
nareshkumarthota Nov 1, 2019
409dabb
Optimize the memory used by the decision table
rameshpolishetti Nov 1, 2019
d01c816
Add support for multiple condition in a decision table cell
rameshpolishetti Nov 3, 2019
710c39f
Adding README for decision table activity
rameshpolishetti Nov 7, 2019
cb71203
Fix README format issue
rameshpolishetti Nov 7, 2019
60b843a
Added formal expression parser
pointlander Dec 3, 2019
af493e6
Improve parser
pointlander Dec 3, 2019
4226a51
Removed license
pointlander Dec 4, 2019
7625da3
Refactor
pointlander Dec 4, 2019
56747f9
upgrade excelize version to latest
rameshpolishetti Dec 6, 2019
bded2de
Merge pull request #83 from project-flogo/feature-decision-table-parser
rameshpolishetti Dec 6, 2019
9c8f217
updated example to use store config file (#81)
nareshkumarthota Dec 6, 2019
97a95d0
Update README.md (#86)
aashish2001 Dec 11, 2019
077f860
Merge branch 'develop' into feature-decision-table
nareshkumarthota Dec 12, 2019
eb1f235
removed redundant examples
nareshkumarthota Dec 12, 2019
d57fcc0
Merge pull request #87 from project-flogo/feature-decision-table
rameshpolishetti Dec 13, 2019
45b1957
Feature travisbuilds (#88)
LakshmiMekala Jan 17, 2020
4fea442
resolving conflicts
nareshkumarthota Mar 17, 2020
c8c8125
fixing replaytplesforrule
nareshkumarthota Mar 18, 2020
e4d69bd
adding replay tuple method in rete
nareshkumarthota Mar 19, 2020
fb4ff64
tuple null handled in redis
nareshkumarthota Mar 19, 2020
4567024
upgrading flogo dependency versions
nareshkumarthota Mar 19, 2020
2bffc7e
go modules updated
nareshkumarthota Mar 19, 2020
973a7f4
resolving conflicts
nareshkumarthota Mar 19, 2020
41ced91
Merge pull request #90 from project-flogo/develop-master
rameshpolishetti Mar 20, 2020
05505b2
added fix for issue 91
nareshkumarthota Mar 24, 2020
2e1fe55
Merge pull request #92 from project-flogo/fix-issue-91
rameshpolishetti Mar 27, 2020
2e3a62f
Feature ruleapi examples (#94)
ykalidin Mar 31, 2020
0a7f842
Feature Decision Table API model changes (#93)
nareshkumarthota Mar 31, 2020
51126dd
Updating examples (#95)
ykalidin Mar 31, 2020
92dca6f
Chore update readme (#96)
ykalidin Apr 1, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 45 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,58 +65,88 @@ First we start off with loading the `TupleDescriptor`. It accepts a JSON string
Next create a `RuleSession` and add all the `Rule`s with their `Condition`s and `Actions`s.

//Create a RuleSession
rs, _ := ruleapi.GetOrCreateRuleSession("asession")
rs, _ := ruleapi.GetOrCreateRuleSession("asession", "")

//// check for name "Bob" in n1
rule := ruleapi.NewRule("n1.name == Bob")
rule.AddCondition("c1", []string{"n1"}, checkForBob, nil)
rule.SetAction(checkForBobAction)
rule.SetContext("This is a test of context")
rule.AddCondition("c1", []string{"n1"}, checkForBob, events)
serviceCfg := &config.ServiceDescriptor{
Name: "checkForBobAction",
Function: checkForBobAction,
Type: "function",
}
aService,_ := ruleapi.NewActionService(serviceCfg)
rule.SetActionService(aService)
rule.SetContext(events)
rs.AddRule(rule)
fmt.Printf("Rule added: [%s]\n", rule.GetName())

// check for name "Bob" in n1, match the "name" field in n2,
// in effect, fire the rule when name field in both tuples is "Bob"
rule2 := ruleapi.NewRule("n1.name == Bob && n1.name == n2.name")
rule2.AddCondition("c1", []string{"n1"}, checkForBob, nil)
rule2.AddCondition("c2", []string{"n1", "n2"}, checkSameNamesCondition, nil)
rule2.SetAction(checkSameNamesAction)
rule2.AddCondition("c1", []string{"n1"}, checkForBob, events)
rule2.AddCondition("c2", []string{"n1", "n2"}, checkSameNamesCondition, events)
serviceCfg2 := &config.ServiceDescriptor{
Name: "checkSameNamesAction",
Function: checkSameNamesAction,
Type: "function",
}
aService2,_ := ruleapi.NewActionService(serviceCfg2)
rule2.SetActionService(aService2)
rule2.SetContext(events)
rs.AddRule(rule2)
fmt.Printf("Rule added: [%s]\n", rule2.GetName())

//Finally, start the rule session before asserting tuples
//Your startup function, if registered will be invoked here

// check for name in n1, match the env variable "name"
rule3 := ruleapi.NewRule("n1.name == envname")
rule3.AddCondition("c1", []string{"n1"}, checkSameEnvName, events)
serviceCfg3 := &config.ServiceDescriptor{
Name: "checkSameEnvNameAction",
Function: checkSameEnvNameAction,
Type: "function",
}
aService3,_ := ruleapi.NewActionService(serviceCfg3)
rule3.SetActionService(aService3)
rule3.SetContext(events)
rs.AddRule(rule3)

//set a transaction handler
rs.RegisterRtcTransactionHandler(txHandler, nil)
//Start the rule session
rs.Start(nil)

Here we create and assert the actual `Tuple's` which will be evaluated against the `Rule's` `Condition's` defined above.

//Now assert a "n1" tuple
fmt.Println("Asserting n1 tuple with name=Tom")
t1, _ := model.NewTupleWithKeyValues("n1", "Tom")
t1.SetString(nil, "name", "Tom")
rs.Assert(nil, t1)

//Now assert a "n1" tuple
fmt.Println("Asserting n1 tuple with name=Bob")
t2, _ := model.NewTupleWithKeyValues("n1", "Bob")
t2.SetString(nil, "name", "Bob")
rs.Assert(nil, t2)

//Now assert a "n2" tuple
fmt.Println("Asserting n2 tuple with name=Bob")
t3, _ := model.NewTupleWithKeyValues("n2", "Bob")
t3.SetString(nil, "name", "Bob")
rs.Assert(nil, t3)

//Now assert a "n1" tuple
t4, _ := model.NewTupleWithKeyValues("n1", "Smith")
t4.SetString(nil, "name", "Smith")
rs.Assert(nil, t4)

Finally, once all `Rule` `Condition's` are evaluated and `Action's` are executed, we can `Retract` all the `Tuple's` from the `RuleSession` and unregister the RuleSession.

//Retract tuples
rs.Retract(nil, t1)
rs.Retract(nil, t2)
rs.Retract(nil, t3)
rs.Retract(nil, t4)

//delete the rule
rs.DeleteRule(rule.GetName())
rs.DeleteRule(rule2.GetName())
rs.DeleteRule(rule3.GetName())

//unregister the session, i.e; cleanup
rs.Unregister()
Expand Down
46 changes: 46 additions & 0 deletions activity/dtable/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Decision Table
This is a `flogo activity` based `Decision Table` implementation used inside a rules application. It can be invoked as rule action service. `Decision Table` provide a tabular way to build complex business rules. Each column can be created based on predefined properties. These properties are defined in the `tuple descriptor` used inside a rule application. Each row can be thought of as one rule in a table made up of many rules. The individual rules are often straightforward, as in the following example.

Rule conditions:
```
person.age > 30
person.gender == "male"
```
Rule actions:
```
application.status = "ACCEPTED"
application.credit = 4000
```
A decision table can consist of hundreds, even thousands of rules each of which is executed only when its specific conditions are satisfied.

## Usage

The available activity `settings` are as follows:

| Name | Type | Description |
|:-----------|:--------|:--------------|
| dTableFile | string | Decision table file path (xlsx & csv extensions are supported) |

A sample `decision table` definition is:
```json
{
"name": "ApplicantSimple",
"description": "Simple Applicants approval dt",
"type": "activity",
"ref": "github.com/project-flogo/rules/activity/dtable",
"settings": {
"dTableFile":"creditcard-file.xlsx"
}
}

```

An example rule action service that invokes the above `ApplicantSimple` decision table is:
```json
"actionService": {
"service": "ApplicantSimple",
"input": {
"message": "test ApplicantSimple"
}
}
```
64 changes: 64 additions & 0 deletions activity/dtable/activity.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
package dtable

import (
"context"

"github.com/project-flogo/core/activity"
"github.com/project-flogo/core/data/metadata"
"github.com/project-flogo/rules/common/model"
"github.com/project-flogo/rules/ruleapi"
)

func init() {
_ = activity.Register(&Activity{}, New)
}

// Activity decision table based rule action
type Activity struct {
dtable ruleapi.DecisionTable
}

// New creates new decision table activity
func New(ctx activity.InitContext) (activity.Activity, error) {
// Read settings
settings := &Settings{}
err := metadata.MapToStruct(ctx.Settings(), settings, true)
if err != nil {
return nil, err
}

// Read decision table from file
dtable, err := ruleapi.LoadDecisionTableFromFile(settings.DTableFile)
if err != nil {
return nil, err
}
// dtable.print()
err = dtable.Compile()
if err != nil {
return nil, err
}
// dtable.print()

// Read setting from init context
act := &Activity{
dtable: dtable,
}
return act, nil
}

// Metadata activity metadata
func (a *Activity) Metadata() *activity.Metadata {
return activity.ToMetadata(&Input{})
}

// Eval implements decision table action
func (a *Activity) Eval(ctx activity.Context) (done bool, err error) {

context := ctx.GetInput("ctx").(context.Context)
tuples := ctx.GetInput("tuples").(map[model.TupleType]model.Tuple)

// evaluate decision table
a.dtable.Apply(context, tuples)

return true, nil
}
179 changes: 179 additions & 0 deletions activity/dtable/activity_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
package dtable

import (
"context"
"testing"

"github.com/project-flogo/core/data/resolve"
"github.com/project-flogo/core/support/test"
"github.com/project-flogo/rules/common/model"
"github.com/stretchr/testify/assert"

"github.com/project-flogo/core/data/mapper"
)

const tupleDescriptor = `[
{
"name":"applicant",
"properties":[
{
"name":"name",
"pk-index":0,
"type":"string"
},
{
"name":"gender",
"type":"string"
},
{
"name":"age",
"type":"int"
},
{
"name":"address",
"type":"string"
},
{
"name":"hasDL",
"type":"bool"
},
{
"name":"ssn",
"type":"long"
},
{
"name":"income",
"type":"double"
},
{
"name":"maritalStatus",
"type":"string"
},
{
"name":"creditScore",
"type":"int"
},
{
"name":"status",
"type":"string"
},
{
"name":"eligible",
"type":"bool"
},
{
"name":"creditLimit",
"type":"double"
}
]
},
{
"name":"processapplication",
"ttl":0,
"properties":[
{
"name":"ssn",
"pk-index":0,
"type":"long"
},
{
"name":"start",
"type":"bool"
}
]
}
]`

var testApplicants = []map[string]interface{}{
{
"name": "JohnDoe",
"gender": "Male",
"age": 20,
"address": "BoltonUK",
"hasDL": true,
"ssn": "1231231234",
"income": 45000,
"maritalStatus": "single",
"creditScore": 500,
},
{
"name": "JaneDoe",
"gender": "Female",
"age": 38,
"address": "BoltonUK",
"hasDL": false,
"ssn": "2424354532",
"income": 32000,
"maritalStatus": "single",
"creditScore": 650,
},
{
"name": "PrakashY",
"gender": "Male",
"age": 30,
"address": "RedwoodShore",
"hasDL": true,
"ssn": "2345342132",
"income": 150000,
"maritalStatus": "married",
"creditScore": 750,
},
{
"name": "SandraW",
"gender": "Female",
"age": 26,
"address": "RedwoodShore",
"hasDL": true,
"ssn": "3213214321",
"income": 50000,
"maritalStatus": "single",
"creditScore": 625,
},
}

func TestNew(t *testing.T) {
err := model.RegisterTupleDescriptors(string(tupleDescriptor))

settings := &Settings{
DTableFile: "test_dtable.csv",
}
mf := mapper.NewFactory(resolve.GetBasicResolver())
initCtx := test.NewActivityInitContext(settings, mf)
act, err := New(initCtx)
assert.Nil(t, err)
assert.NotNil(t, act)
}

func TestEval(t *testing.T) {
err := model.RegisterTupleDescriptors(string(tupleDescriptor))

settings := &Settings{
DTableFile: "test_dtable.xlsx",
}
mf := mapper.NewFactory(resolve.GetBasicResolver())
initCtx := test.NewActivityInitContext(settings, mf)
act, err := New(initCtx)
assert.Nil(t, err)
assert.NotNil(t, act)

tuples := make(map[model.TupleType]model.Tuple)
tuple, err := model.NewTuple(model.TupleType("applicant"), testApplicants[0])
assert.Nil(t, err)
assert.NotNil(t, tuple)
tuples[tuple.GetTupleType()] = tuple

tc := test.NewActivityContext(act.Metadata())
tc.SetInput("ctx", context.Background())
tc.SetInput("tuples", tuples)
act.Eval(tc)

expectedStatus, err := tuple.GetString("status")
assert.Nil(t, err)
assert.Equal(t, "VISA-Granted", expectedStatus)
expectedEligible, err := tuple.GetBool("eligible")
assert.Nil(t, err)
assert.Equal(t, true, expectedEligible)
expectedCreditLimit, err := tuple.GetDouble("creditLimit")
assert.Nil(t, err)
assert.Equal(t, 2500.0, expectedCreditLimit)
}
Loading