Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maraboupy core tests #287

Merged
merged 2 commits into from
Jun 16, 2020
Merged

Conversation

kjulian3
Copy link
Collaborator

The file maraboupy/tests.py was moved and refurbished to create the test_core.py, which tests that Marabou can be called from python by interacting directly with MarabouCore. The test can be run with only pytest installed. Moving MarabouNetworkNNet import into a try block allows Maraboupy to work even when numpy is not installed.

This is a starting point for testing MarabouCore, but there are additional functions not yet tested, such as

  • DNC mode
  • createInputQuery()
  • Additional PL constraints, such as absolute value and disjunction
  • Statistics methods beyond hasTimedOut()

In addition, MarabouCore.cpp is not yet part of code coverage

@kjulian3 kjulian3 requested a review from yuvaljacoby June 14, 2020 22:15
Comment on lines 59 to 60
ipq.setLowerBound(1, 1)
ipq.setUpperBound(1, 2)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be 0,1 ? otherwise you set x and relu(x) to be always 1

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would make more sense, and I changed it to 0, LARGE because the upper bound comes from the upper bound on the first variable, which Marabou should propagate. I've reformatted the tests a little bit to also check that Marabou provides a default bounds when none are provided, and that bound propagation tightens these bounds as well.

Copy link
Collaborator

@yuvaljacoby yuvaljacoby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

Do you understand why codecov marks it as fail?

@kjulian3
Copy link
Collaborator Author

Yeah, codecov is mad that I didn't write a test for the small change I made to Marabou.py. To test that part of the code, we'd have to run tests without numpy installed. I've tested it on my machine, and I don't think it's worth trying to configure tests to run without numpy to show it works.

@kjulian3 kjulian3 merged commit cb7c8e5 into NeuralNetworkVerification:master Jun 16, 2020
AleksandarZeljic pushed a commit to AleksandarZeljic/Marabou that referenced this pull request Oct 9, 2020
* Adding core tests

* Updates to testing
matanost pushed a commit that referenced this pull request Nov 2, 2021
* Adding core tests

* Updates to testing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants