Skip to content

Latest commit

 

History

History
30 lines (18 loc) · 2.54 KB

2.9. Usage - Fully Automated Testing of Controllers.md

File metadata and controls

30 lines (18 loc) · 2.54 KB
Raw

Testing Controllers

Fully Automated Controller Tests

The testing tool supports fully automated testing of Controllers by replacing the default Testing Façade application with your own application implementing the Question API to receive and process the requests from the NMOS Testing Tool. A complete example utilising Flask and Selenium to test the Sony nmos-js Controller can be found in the sony/nmos-js repository.

Building your fully automated test interface

The Testing Façade IP/Hostname and Port on the test suite should be set to the location of your application. To carry out each test POST requests are sent from the Test Suite to the Question API endpoint http://<ip address or hostname>:<port>/x-nmos/testquestion/<version>.

On receiving a POST request from the Testing Tool, your application uses the question_id to instruct your chosen automated test framework (e.g. Selenium) to carry out the required tasks on your Controller and determine appropriate answers.

Your application POSTs the question_id and answer(s) to the Answer API endpoint given in answer_uri.

JSON schema and RAML files for both the Question and Answer API endpoints can be found in the testingfacade folder. An example python class for storing and retrieving the Question data can be found at testingfacade/DataStore.py

Some tests contain multiple questions, all of which must be answered correctly to pass the test.

There are three types of question that need different kinds of answer:

  • single_choice - report a single answer_id from the answers object
  • multi_choice - report a list of multiple answer_id's from the answers object
  • action - no answers given, simply report null when ready to move to the next question

Example JSON files for each type of Question and Answer that could be sent and received can be found in the examples folder.

On receiving a POST request to the Answer API, the test suite will process the answer and send the next question to the Question API. If the Answer API does not receive a request within the timeout period, the current test will expire and the first question of the next test will be sent.

The test suite begins with a pre_tests_message and ends with a post_tests_message. These are action questions so the answer_response is null to start and end the actual tests.