Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix bug in dnc evaluate #333

Merged
merged 3 commits into from
Jul 28, 2020
Merged

Conversation

wu-haoze
Copy link
Collaborator

Fixing issue #330.
The issue is that when some worker finds a SAT instance and returns a mapping from variable indices to assignments, it is using variable indices after preprocessing. This is because the worker itself does not preprocess the query but directly take in the preprocessed query, so it does not store the mapping from original variable index to the new index.
This information is stored in DnCManager._baseEngine. Therefore, we copy the tableau state of DnCManager._engineWithSATAssignment into DnCManager._baseEngine and extract solutions from the latter object, instead of the former object.

@wu-haoze wu-haoze requested a review from kjulian3 July 24, 2020 23:31
@wu-haoze wu-haoze linked an issue Jul 24, 2020 that may be closed by this pull request
@kjulian3
Copy link
Collaborator

We need to add tests for DNC mode in Maraboupy. I tried adding one just now, and ran into an unexpected error. Here's the test I was trying to run:

# Tests Marabou's DNC option
import pytest
from .. import Marabou
import os
import numpy as np

# Global settings
OPT = Marabou.createOptions(verbosity=2, dnc=True, numWorkers=2) # Turn off printing, turn on DNC with two workers
TOL = 1e-6                                                       # Set tolerance for checking Marabou evaluations
NETWORK_FOLDER = "../../resources/nnet/acasxu"                   # Folder for test networks

def test_dnc_unsat():
    """
    Test the 1,1 experimental ACAS Xu network. 
    Evaluate a simple local robustness property for a network with DNC mode activated
    """
    filename =  "ACASXU_experimental_v2a_1_1.nnet"
    filename = os.path.join(os.path.dirname(__file__), NETWORK_FOLDER, filename)
    network = Marabou.read_nnet(filename)
    
    # Local robustness input constraint
    centerPoint = [-0.2454504737724233, -0.4774648292756546, 0.0, -0.3181818181818182, 0.0]
    for var, val in zip(network.inputVars[0], centerPoint):
        network.setLowerBound(var, val - 0.002)
        network.setUpperBound(var, val + 0.002)

    # Solve with no output constraint
    # Should return SAT, and vals should be a populated dictionary
    vals, stats = network.solve(options = OPT, filename = "", verbose=True)
    assert len(vals) > 0

I created a new file with this code in it (maraboupy/test/test_dnc.py), and calling pytest via pytest test/test_dnc.py from the maraboupy directory will produce the Marabou Error, code 6.

@wu-haoze wu-haoze merged commit 32ece55 into NeuralNetworkVerification:master Jul 28, 2020
@wu-haoze wu-haoze deleted the dnc-eval branch July 28, 2020 22:59
matanost pushed a commit that referenced this pull request Nov 2, 2021
* fix bug in dnc evaluate

* only copy over tableau state

* Add tests for DNC mode

Co-authored-by: Kyle Julian <kjulian3@stanford.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

evaluateWithMarabou doesn't work with DNC
2 participants