Skip to content

IPTS ‐ HOW TO ADD TEST SUPPORT

Jump to bottom
Davide Tomé edited this page May 24, 2024 · 4 revisions

1 Document Scope

ℹ️ This document explains how to add test support for a new board in IPTS test suite

2 Requirements

2.1 Software Requirements

  • PC with Windows 10 or higher
  • Visual Studio 2019 or higher
  • MS .NET Framework ver. 4.7.2
  • Inno Setup Windows installer
  • ESD CAN/USB SDK
  • IPTS test suite repository

3 IPTS Visual Studio project overview

The IPTS is a Visual C# project based on the MS .NET Framework, its folder contains :

3.1 iCubProductionTestSuite.sln

This is the Visual Studio solution file, double click to open the project and browse the source code

3.2 ipts.iss

This is the Inno Setup project file, it is used to generate the Windows installer file

3.3 Output

This is the folder where the installer generated by InnoSetup is stored

3.4 iCubProductionTestSuite

This is the IPTS project folder containing all required files and folders

Here is a brief description of the most important files/folders :

  • ipts.xml : is the file where you can add/remove or modify boards to be tested and its tests
  • classes (folder) : contains all the custom C# classes used by the project
  • bin (folder) : contains the binaries produced by the compiler
  • tools (folder) : contains external tools to be installed in the installation process, the batch files used to flash the boards under test, the test firmware binaries, batch files to automatically update the icub-firmware-build repo to flash official binaries at the end of the test.
  • img (folder) : contains all the images used by the GUI and configured in the ipts.xml file
  • TestReports (folder) : contains the test reports files

4 Add a new board test support

To add a new board test support we need to :

  1. Know the interface type used to communicate with the test FW (CAN or SERIAL are supported)
  2. Know the commands expected by the test FW
  3. Modify the ipts.xml file accordingly
  4. Add pictures
  5. Add batch files
  6. Add .hex test FW file
  7. Add the board to the setup-fw-repo.bat script
  8. Produce the new installer
  9. Produce a new IPTS release on Github

4.1 Interface type used to communicate with the test FW

The interface is used by IPTS to communicate with the test FW and it is declared in the test plan section of the board in the ipts.xml file.

Here is an example of the interface declaration for MTB4C board :

	<testplan id="1" board="MTB4C" iitcode="12008.C" reportsDir="mtb4c" rev="1">

		<interfaces>
			<interface name="CAN" netPort="0" bitrate="1000" messageID="0x551" />
		</interfaces>

At the time being the boards tested with IPTS communicate with it via ESD CAN/USB interface; the test FWs are all structured to wait for a CAN frame, parse it and execute the related test.

At the moment SERIAL interface is used only to test the serial interface of the RFE_MASTER board.

4.2 Commands expected by the test FW

Here is an example of the commands expected by the MTB4 test FW using the CAN interface:

4.3 Modify the ipts.xml file

At this stage, once we know the interface used to communicate with the board and commands expected by the test FW, we have to add a testplan section dedicated to the new board in the ipts.xml file.

  <testplan id="0" board="MTB4" iitcode="12008.B" reportsDir="mtb4" rev="5">

    <interfaces>
      <interface name="CAN" netPort="0" bitrate="1000" messageID="0x551" />
    </interfaces>

    <test id="1" name="Test Assenza corto 5V-GND" stopOnFail="true">
      <operation type="okDialog" text="Verificare che i pin 1 e 4 del connettore J1 non siano in corto." image="img/mtb4/MTB4_short.jpg" />
      <operation type="passFailDialog" text="I pin NON SONO in corto?" />
    </test>

    <test id="2" name="Flash Firmware di test" stopOnFail="true">
		  <operation type="okDialog" text="Collegamenti setup:

1. Connettere l' ESD CAN-USB (2) ed il ST-Link (3) alle USB del PC (1)

2. Connettere l' ESD CAN-USB (2) alla MTB4 (MTB4-J1) 

3. Connettere la “ADC_MTB_LED” (6) alla MTB4 (MTB4-J3) 

4. Premere Invio a programmazione terminata
					
UTILIZZARE SEMPRE IL TOOL PER ESTRARRE I CONNETTORI MOLEX (5)"
					   
					 image="img/mtb4/MTB4_setup.jpg" />
		  <operation type="batch" batchDir="tools\boards\mtb4\flash" batchFile="mtb4_test.bat" />
	  </test>

    <test id="3" name="Test comunicazione CAN" stopOnFail="true">
      <operation type="send" interface="CAN" command="06"/>
      <operation type="receivePassFail" interface="CAN" valPass="0xAA"/>
    </test>
    
    <test id="4" name="Verifica Revisione Firmmware di Test" stopOnFail="true">
      <operation type="send" interface="CAN" command="00" />
      <operation type="receivePassFail" interface="CAN" valPass="0x03 0x00" logMess="1" />
    </test>

    <test id="5" name="Test Led OFF">
      <operation type="send" interface="CAN" command="01" />
      <operation type="yesNoDialog" text="I led rosso e blu sono spenti?" image="img/mtb4/MTB4_ledoff.jpg" />
    </test>

    <test id="6" name="Test Led ON">
      <operation type="send" interface="CAN" command="02" />
      <operation type="yesNoDialog" text="I led rosso e blu sono accesi?" image="img/mtb4/MTB4_ledon.jpg" />
    </test>

    <test id="7" name="Test Temperatura">
      <operation type="inputValue" text="Inserire il valore della temperatura ambiente" var="tempIn" varType="num" />
      <operation type="send" interface="CAN" command="03" appendVar="tempIn" />
      <operation type="receivePassFail" interface="CAN" valPass="0xAA" logMess="1" />
    </test>

    <test id="8" name="Test GPIO">
      <operation type="okDialog" text="Verificare che i sei led della scheda ADC_MTB4_LED_12660_A si accendano in sequenza..." image="img/mtb4/MTB4_gpio.jpg" />
      <operation type="send" interface="CAN" command="05" />
      <operation type="wait" value="4000" />
      <operation type="passFailDialog" text="i 6 led si sono accesi in sequenza?" />
    </test>

    <test id="9" name="Test IMU">
      <operation type="send" interface="CAN" command="04" />
      <operation type="receivePassFail" interface="CAN" valPass="0xA0"  logMess="1" />
    </test>

    <test id="10" name="Test secondo CAN">
      <operation type="okDialog" text="Connettere il cavo CAN al secondo connettore (J2) e premere ok" image="img/mtb4/MTB4_can.jpg" />
      <operation type="wait" value="1000" />
      <operation type="send" interface="CAN" command="06" />
      <operation type="receivePassFail" interface="CAN" valPass="0xAA" />
    </test>

    <test id="11" name="Flash Bootloader/Applicativo">
      <operation type="batch" batchDir="tools\boards\mtb4\flash" batchFile="mtb4_prod.bat" />
    </test>
  </testplan>
Overview of the main tags used

Here a description of the principal tags used.

4.3.1 testplan 
 	<testplan id="0" board="MTB4" iitcode="12008.B" reportsDir="mtb4" rev="5">

It contains all the tests to be performed on the board, the required fields are :

  • id : this is the ID of the test plan; pay attention to making them sequential; if you add the test plan at the bottom you just increase by one the previous test plan ID.
  • board : the name of the board
  • iitcode : the IIT code of the board (Wingst)
  • reportsDir : the folder under the TestReports folder where the test reports will be saved
  • rev : the revision of the test plan
4.3.2 interface 
    <interfaces>
      <interface name="CAN" netPort="0" bitrate="1000" messageID="0x551" />
	<interface name="SERIAL" netPort="COM4" bitrate="9600" />
    </interfaces>

It defines the properties of the communication interface used by the test FW:

  • name : can be CAN (using the ESD CAN/USB device) or SERIAL
  • netport : the name of the CAN net (i.e "0") or the serial COM PORT (i.e. "COM4")
  • bitare : the bitrate of the CAN BUS or the baud rate of the SERIAL port
4.3.3 test 
    <test id="4" name="Verifica Revisione Firmmware di Test" stopOnFail="true">
      <operation type="send" interface="CAN" command="00" />
      <operation type="receivePassFail" interface="CAN" valPass="0x03 0x00" logMess="1" />
    </test>

It defines the test to be executed and wraps the necessary operations :

  • id : the unique sequential id of the tests in the test plan
  • name : the name of the test; it will be displayed in the GUI and referenced in the test reports
  • stopOnFail : it instructs the test suite to stop the run on test failure; it may be useful in case of critical tests (i.e. short circuit check); defaulkt is false
4.3.4 operation 
    <test id="4" name="Verifica Revisione Firmmware di Test" stopOnFail="true">
      <operation type="send" interface="CAN" command="00" />
    </test>

It is the tag which instructs the test suite about which operation has to be performed; the principal types can be parsed are :

  • send : this sends a command to the test FW via the declared interface :
	 <operation type="send" interface="CAN" command="00" />
  • receivePassFail : it waits for a response from the test FW and compares it with the declared ValPass value to determine if the test is PASS or FAIL; logMess if put to 1 (default is 0) reads the other bytes of the CAN frame and print them in the report (in case the test gives also some values) :
    <operation type="receivePassFail" interface="CAN" valPass="0xAA" logMess="1" />
  • okDialog : it is meant to give the operator an instruction to prepare for the next test; it shows a MessageBox with a message an OK button and eventually a picture :
    <operation type="okDialog" text="Verificare che i pin 1 e 4 del connettore J1 non siano in corto." image="img/mtb4/MTB4_short.jpg" />
  • passFailDialog : it is meant to ask the operator for a physical event that can't be checked automatically (i.e. LED blinking); it shows a MessageBox with a message and YES/NO buttons and eventually a picture :
   <operation type="passFailDialog" text="Scollegare il cavo USB, Il led Blue é sempre acceso?"  image="img/rfe/rfe-j5_2jpg"/>
  • wait : it instructs the test suite to wait X milliseconds before executing the next operation 
   <operation type="wait" value="4000" />
  • batch : It runs a batch file; usually used for flashing test FW and official bootloader and application at the end of a successful test
  <operation type="batch" batchDir="tools\boards\mtb4\flash" batchFile="mtb4_test.bat" />

4.4 Add pictures

The pictures declared in the ipts.xml file have to be stored under the img folder inside the related board folder

4.5 Add batch files

The batch files used for flashing the test FW at the beginning and the official bootloader and application of the board at the end of the test are located under the tools/boards folder inside the related board folder.

They use the STM32 ST-LINK Utility v4.5.0 (the installer is included in the Windows installer of IPTS) CLI to flash the uP using an ST-LINK debugger.

4.6 Add .hex test FW file

The .hexfile of the test FW declared in the batch script has to be stored under the flash/hex folder inside the board folder, i.e. for MTB4 :

4.7 Add the board to the setup-fw-repo.bat script

The batch files which flash the official bootloader and application at the end of the test plan are retrieved in icub-firmware-build folder for which is made a sparse checkout at the first run of IPTS after the installation; then the repo will be updated at each run of the test suite.

Then we have to add the board in the script which do the actions described above :

4.8 Produce the new installer

In order to produce the new Windows installer we need to :

  1. Update the IPTS revision
  2. Compile the solution to generate the new executable
  3. Add new files to the InnoSetup script
  4. Generate the Windows installer
  5. Open a pull request to merge the changes upstream

4.8.1 Update the IPTS revision

In order to update the IPTS revision we have to modify this line

Usually, the versions change as follows:

  • Major : if there are changes to the source code
  • Minor : if we are adding a new test support for a new board/motor
  • Patch : if we do minor fixes like adding/removing a test to an existing board

In this case, the revision should become 1.6.0

4.8.2 Compile the solution to generate the new executable

Now we compile the solution in Release mode :

4.8.3 Add new files to the InnoSetup script

Now we have to update the revision and include all the new files (images, board folders, batch files...) to the InnoSetup script :

To update the revision in the script, just change this line accordingly the revision set in the IPTS source code.

To add folders, files and so on you have to add them in the Files section :

To add necessary third-party installers to be included and launched by the Windows installer of IPTS, you have to add them in the Run section :

4.8.4 Generate the Windows installer

To generate the new Windows installer of IPTS just click on the compile icon :

If it compiles successfully you'll find the new installer in the Output folder.

4.8.5 Open a pull request to merge the changes upstream

Finally, open a PR on the upstream repo

4.9 Produce a new IPTS release

In order to produce a new GitHub release of IPTS :

4.9.1 Go to the Releases section of the repo

4.9.2 Click on Draft a new release

4.9.3 Fill in the Release title and body following the previous formats

4.9.4 Add attachments to the release

Needed attachments are the new installer, all test procedures available in the previous release adding the on related to the new board (for creating it refer to the existing ones i.e. MTB4)

4.9.5 Tag the new release (must be the same as the IPTS revision)

⚠️ The link to the new release should be provided to the supplier who will test the new boards

Clone this wiki locally