Code design

NSTAT leverages the conciseness and flexibility of Python to quickly prototype and integrate new functionality. Its design approach aims at not centralizing the overall test complexity within a single entity, but distributing it across:

• the test configuration
• the test code itself
• the network controllers/emulators (northbound/southbound) handling logic
• the network monitor handling logic

The current NSTAT version has strived to keep most test cases as generic and independent from specific implementations as possible, yet there are some cases that have been developed with specific controller implementations in mind and are therefore tightly coupled to them. Specifically, many test cases make use of functionality and APIs provided by the OpenDaylight controller, such as acquiring information from its operational DataStore or changing the statistics requesting interval.

Currently NSTAT provides handling logic for the following components:

• Supported SDN controllers. Currently only OpenDaylight controller is supported.
• MT-Cbench emulator
• Multinet emulator
• NorthBound emulator
• oftraf OpenFlow messages monitor tool

Each item from the previous list is represented in NSTAT with a class. These classes are organizes hyerarchical from abstract classes, superclasses, to more specific. For example we have a Controller superclass that is extended from all classes, subclasses, that represent a controller, such as OpenDaylight, ODL Controller, or ONOS Controller. Currently we support only OpenDaylight.

Controller/Emulators handling logic

Additionally to have a more abstract level of implementation, we have a common handling interface for all controllers, implemented as a set of handlers that are invoked from the Controller superclass and subclasses. In this way for different versions of the same controller (our expectation is to use the same class implementation, even for different Controllers and transfer common methods to the superclass. We have not make this investigation yet) we do not have to implement different Controller subclasses. The only change is the implementation of the set of Handlers.

There is a common API in all NSTAT nodes handling logic (SouthBound/NorthBound emulator node), Controller node. Next, we describe this common API logic for all handlers. Since all NSTAT nodes (Controllers, SouthBound, NorthBound emulators) are stored and maintained in different repositories, NSTAT just needs to get this software on its own workspace and clean it when the test execution is over. For that reason, under /controllers/... and /emulators/... directories all necessary build/clean handlers are stored

1. Build handler: ./build.sh
   • expected behaviour: builds the controller
   • prereqs: should be executable
   • input arguments: none
   • exit status: 0 in case of success, non-zero otherwise
2. Clean handler: ./clean.sh
   • expected behaviour: starts the controller so that a switch can connect to it
   • prereqs: should be executable
   • input arguments: none
   • exit status: 0 in case of success, non-zero otherwise

The above design described for Controllers is followed for any other NSTAT component. A more detailed diagram about how different classes of NSTAT are related is presented in the NSTAT class diagram.

Code structure

Path	Description
controllers/	handlers for various controller implementations
controllers/<controller>/	handlers for the <controller> defined in folder name. Contains the build and clean handlers, that clones the nstat-sdn-controllers and cleans up the folder to its initial state respectively.
deploy/	deployment methods of NSTAT testing environment
deploy/docker	deployment of NSTAT testing environment, with docker containers
deploy/docker/proxy	docker containers with proxy support, in case host sits behind a proxy
deploy/docker/no_proxy	docker containers without proxy support
docs/	automated documentation builder, ReadTheDocks
emulators/	handlers/implementation for various SB/NB emulators or generators
emulators/mt_cbench/	handlers for MT-Cbench
emulators/multinet/	handlers and implementation for Multinet custom topologies
emulators/nb_generator/	handlers and implementation for NB flow generator
jenkins_jobs/	Jenkins jobs builder (JJB) jobs specifications
monitors	monitoring tools of controller interfaces. Currently there is only one monitoring tool, oftraf (see next for details)
monitors/oftraf/	handlers for oftraf. Statistic report tool which filters OpenFlow packets from a network interface
stress_test/	test implementations and configurations
stress_test/sample_test_confs/	JSON configuration files for sample functional end to end tests
stress_test/sample_test_confs/<controller>/	JSON configuration files for sample functional end to end tests using defined in folder name
stress_test/stress_test_confs/	JSON configuration files for large-scale stress tests
stress_test/stress_test_confs/<controller>/	JSON configuration files for large-scale stress tests using defined in folder name
util/	generic Python utility functions
util/unitests/	unit tests for generic Python utility functions

Test implementation under stress test folder

NSTAT supports the stress test scenarios presented at the main wiki. All tests scenarios are implemented within test_run.py, as methods of the TestRun class.

Upon initialization of a TestRun all necessary objects for the specific test are created and setup of initial testing conditions takes place.

The initiation of TestRun instance and the execution of the desired test scenario, is done in test_type.py, test_selector() method of TestType class. The selection is done according to the input is provided when running NSTAT. For more details see NSTAT workflow and usage