Write, test and repeat using YAML language

Highlights

• Write automated tests in plain Yaml
• Design visual tests in a matter of minutes
• Use OpenCV and/or Bitmap comparison to perform visual tests
• Scale up browser compatibility checks
• Of course, hassle-free installation

Installation

IMPORTANT: Given this package is a node project, let's ensure to install node and npm as pre-requisites before setting up the project. For further info.

npx setup-kanstructor demo-project

Running the above command automatically sets up project structure, along with example files. This section provides the details on how the project resources are organised. Note that running this command also requires you to have git installed , if not install from here.

In order to run the tests, the following command does the job:

cd demo-project
npm run test

video-example.mp4

Quick Start

• Step 1 : The src folder under the project is going to be a root directory for all testing stuff, and the most of the project contents will be inside resources directory.

• Step 2 : Under resources folder, let's create tests folder which will contain test files in a plain YAML format. Note that, this package kanstructor identifies a file as a test file only if it ends in *test.yaml. More on how to write tests?

• Step 3 : Many a times, CSS and XPath values used to identify html elements will be used in several places across test files. To keep all such values in centralized place, the folder elements needs to be created within resources. Just like test files, the element files need to be ending in *element.yaml The following snippet shows some example element yaml.

   ### Login page elements' xpath and css
  
   login_email: "[name='email']"
   login_password: "[name='password']"
   login_button: "//button[normalize-space()='Login']"
   home_logo: "a[href*='home']"

• Step 4 : The next step is, the folders extracted-contents and snapshots need to be created to save all contents extracted during testing to external files and to keep baseline screenshots that will be verified against app under tests during testing respectively.

• Step 5 : All common configurations such as browser, env etc will have to be in the file: config.yaml under config folder. The following snippet shows some examples.

    browser: chrome
    headless: false
    device: Desktop Chrome
    url: https://github.com/5v1988/kanstructor

  In addition, there is also a way to configure options required to perform visual comparison of snapshots through the file visual.tests.config.yaml under config folder, and the following snippet shows the basic configurations that are generally used.

  output:
    errorColor:
      red: 255
      green: 0
      blue: 0
    errorType: flat #"flat" or "movement" or "flatDifferenceIntensity" or "movementDifferenceIntensity" or "diffOnly"
    transparency: 0.3
    largeImageThreshold: 1200
    useCrossOrigin: false
    boundingBoxes:
      - left: 0
        top: 0
        right: 1300
        bottom: 800
  returnEarlyThreshold: 0
  scaleToSameSize: false
  ignore: antialiasing # "nothing" or "less" or "antialiasing" or "colors" or "alpha";

• Step 6 : Lastly, to run all tests, the test runner runMe.js needs to be created in the project as follows:

    import runMe from 'kanstructor'
    runMe();

Now execute tests using node src/runMe.js ( or npm run test) from command line. Note that, not necessarily that the runner method must always be named as runMe; Once all setup is complete, the below is the expected project structure

.
├── README.md
├── node_modules
├── package-lock.json
├── package.json
└── src
    ├── resources
    │   ├── config
    │   │   └── config.yaml
    |   │   └── visual.tests.config.yaml
    │   ├── elements
    │   │   └── todo-element.yaml
    │   ├── reports
    │   │   ├── results.html
    │   │   └── results.json
    │   ├── snapshots
    │   │   ├── original-screenshot-1.png
    │   │   └── reference-screenshot-1.png
    │   └── tests
    │       └── todo-test.yaml
    └── runMe.js

Write Tests

— Tests are expected to be written in Yaml files, otherwise known as test files while using this package. Each of these tests should have to be written using 3 A's of testing: Arrange-Act-Assert

description: Some tests on cypress todo demo site
tests:
 - name: Set and delete todo lists
   exclude: false

   arrange:
     - name: Open the url for the app under test
       action: openUrl
       url: url

     - name: Set value for the first item in storage
       action: setValue
       key: firstItem
       value: Schedule doctor appointment

     - name: Set value for the second item in storage
       action: setValue
       key: secondItem
       value: Prepare a blog content

   act:
     - name: Add the first item
       id: 10001
       role: textbox
       text: What needs to be done?
       action: type
       value: ${firstItem}

     - name: Press Enter
       pause: 1
       action: press
       value: Enter

     - name: Add the second item
       locator: .new-todo
       action: type
       value: ${secondItem}

     - name: Press Enter
       pause: 1
       action: press
       value: Enter

     - name: Add the third item
       refId: 10001
       value: Fix the air conditioner

     - name: Press Enter
       pause: 1
       action: press
       value: Enter

     - name: Screenshot after adding all items
       pause: 1
       action: snapshot
       path: "src/example/resources/snapshots/original-screenshot-1.png"    

     - name: Hover to the first item
       text: ${firstItem}
       action: hover

     - name: Delete the first item
       pause: 2
       locator: "//div[normalize-space()='Schedule doctor appointment']//button"
       action: click

     - name: Hover to the second item
       text: ${secondItem}
       action: hover

     - name: Delete the second item
       pause: 2
       locator: "//div[normalize-space()='Prepare a blog content']//button"
       action: click

     - name: Hover to the third item
       text: Fix the air conditioner
       action: hover

     - name: Delete the third item
       pause: 2
       locator: "//div[normalize-space()='Fix the air conditioner']//button"
       action: click

     - name: Screenshot after deleting all items
       pause: 1
       action: snapshot
       path: "src/example/resources/snapshots/original-screenshot-2.png"

   assert:
     - name: Verify if the first item deleted
       pause: 2
       type: standard
       text: ${firstItem} 
       state: invisible

     - name: Verify if the second item deleted
       pause: 2
       type: standard
       text: ${secondItem}
       state: invisible

     - name: Verify if the third item deleted
       pause: 2
       type: standard
       text: Fix the air conditioner
       state: invisible

     - name: Compare screenshot after all items added
       type: snapshot
       original: "src/example/resources/snapshots/original-screenshot-1.png"
       reference: "src/example/resources/snapshots/reference-screenshot-1.png"
       tolerance: 1

 # Visual comparison using OpenCV
     - name: Compare screenshot after all items deleted by OpenCV
       type: glancing
       original: "src/example/resources/snapshots/original-screenshot-2.png"
       reference: "src/example/resources/snapshots/reference-screenshot-2.png"
       tolerance: 3

Guidelines

— A test file can have more than one test, however, our recommendation is to have a few of them, organized by some commonalities

— A test folder tests can contain several test files; No limits

— The high-level blocks — Arrange, Act and Assert, contain a sequence of steps to perform certain actions during testing.

Locating Strategy

This package allows to locate page elements using their accessible name and implicit role. For instance, in the below example, the textbox is located using its placeholder text What needs to be done?.

- name: Add the first item
  id: 10001
  role: textbox
  text: What needs to be done?
  action: type
  value: ${firstItem}

It's also important to that role must always go with text, and at the moment, the other supported roles are as follows:

• textbox
• checkbox
• radio
• link
• option
• button
• slider
• switch

Alternatively, If you are well versed in writing Xpath or CSS, you can simply use locator attribute in the blocks.

Reusing Blocks

By setting id for a test block, it will become reusable and can be used again within the same test or even in the test under the different yaml file. This is done using refId when needed.

- name: Add the first item
  id: 10001
  role: textbox
  text: What needs to be done?
  action: type
  value: ${firstItem}

In the above example, the id is set as 10001. So, by using this id as refId, this test block can be re-used as follows:

- name: Add the third item
  refId: 10001
  value: Fix the air conditioner

While doing so, all attributes except name and value will be taken up from the original test block.

State management

Sometimes, the test data either static or dynamic will have to be shared among blocks. This can be achieved by setting such values with an action named setValue so they can be accessed later part of the tests using ${variableName} or $variableName.

- name: Set value for the first item in storage
  action: setValue
  key: firstItem
  value: Schedule doctor appointment

As you can see from this example, the value for the key firstItem is set-up once and it can be accessed anywhere else later with text(or value) later as shown below.

- name: Verify if the first item deleted
  pause: 2
  type: standard
  text: ${firstItem} 
  state: invisible

Block Reference

Arrange

Action	Description	Keys	Example
openUrl	Open an app url in browser	Required — name, url Optional — pause	- name: openUrl url: https://github.com/5v1988
setValue	Set test data in value storage	Required — key, value	- name: Set value for the username action: setValue key: firstItem value: 5v1988

Act

Action	Description	Keys	Example
type	Enter characters into textboxes	Required — name, action, locator, value Optional — pause	- name: Type in username action: type locator: "input[name='email']" value: 5v1988@gmail.com
check, uncheck	Check (or Uncheck) radio button/checkbox	Required — name, action, locator, Optional — pause	- name: Choose a gender action: check locator: "input[type='checkbox']"
click, doubleclick	Click (or Doubleclick) button/link	Required — name, action, locator, Optional — pause	- name: Type in username action: click locator: '#file-submit'
select	Select a dropdown value	Required — name, action, locator, value Optional — pause	- name: choose_dropdown locator: "#dropdown" action: select value: Option 2
press	Simulate a key press	Required — name, action, value Optional — pause	- name: Press enter action: press value: Enter
clear, focus, hover	Clear (or focus or hover) on html element	Required — name, action, locator Optional — pause	- name: hover on the login link locator: "//button[@id='login']" action: hover
snapshot	Take a screenshot of a current window	Required — name, action, path Optional — pause	- name: Screenshot the login failure pause: 1 action: snapshot path: "path/to/save.png"
upload	Upload a file to the app	Required — name, action, locator path Optional — pause	- name: Upload an image action: upload locator: '#file-upload' path: src/example/innerText.txt
extract	Extract text contents from the current window. Page source of the current window will be extracted by default and locator attribute is mandatory if `extractType` is given	Required — name, action, path Optional — locator extractType — textContents, innerText, innerHTML pause	- name: Extract form contents action: extract path: "path/to/save.txt" locator: "form#customer" extractType: innerText
setValue	Set test data in value storage	Required — key, value( or locator)	- name: Set value for the username action: setValue key: firstItem value: 5v1988

Assert

Type	Description	Keys	Example
standard	Assert the expectation by using page element(s)	Required — name, type, locator, or (role or/and text) role (always used with 'text'), text, state (accepted values: visible, invisible, enabled, disabled, checked, unchecked, containText), Optional — pause	- name: Verify dropdown selected type: element locator: "//option[@selected]" state: containText text: Option 2
compare	Assert if text matches with value storage	Required — key, locator (or value or text), Optional — pause	- name: Compare username key: ${username} text: 5v1988
snapshot	Compare the expected screenshot with the actual one on the current screen	Required — name, type, original, reference, tolerance (lies between 0 (to be exact) and 100 (ignore comparison)) Optional — pause	- name: Verify failure screen type: snapshot original: "path/to/screenshot.png" reference: "path/to/reference.png" tolerance: 1

Roadmap

— [X] Browser
— [X] Summary report
— [O] Parameterized tests
— [O] Mobile

Contributing

We really appreciate and value the work that you do for this project. Needless to say, contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make will benefit everybody else and are greatly appreciated.

Please read our contribution guidelines, and thank you for being involved!

Code of Conduct

See the Code of Conduct for details. Basically it comes down to:

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.

Support

https://discord.gg/GWfMu5Cwq6

License

This project is licensed under the GPLv3 license.

See LICENSE for more information.

Happy Testing!