Skip to content

Latest commit

 

History

History
256 lines (168 loc) · 6.51 KB

README.sh.md

File metadata and controls

256 lines (168 loc) · 6.51 KB
Raw

This is source file for README.md

DO NOT TRY TO RUN.

Sorry but it may use a 'proprietary tools'

I have no time to polish and publish.

echo "

ui-testing 2019

After 4 years I have returned to the same mission and it is time to summarize current stutus. In the start of year 2015 I have reviewed 11 frameworks that I considered significant players, so lets see what is the status now. Old review is here on 2015 branch.

Quick look at npmtrends and statistics:

https://www.npmtrends.com/selenium-webdriver-vs-browserstack-webdriver-vs-wd-vs-webdriverio-vs-kommando-vs-nightwatch-vs-karma-vs-leadfoot-vs-intern-vs-jscoverage

Discontinued frameworks are:

  • browserstack-webdriver
  • kommando
  • blanket
  • jscoverage

Wd stayed stucked with Selenium 2 version and JsonWireProtocol (which is obsolete) and now we have selenium version 4 see https://techbeacon.com/app-dev-testing/selenium-30-40-50-roadmap-finally-unveiled for info about changes.

So we end up with half players out of game, just after 4 years.

2019 choices

  • selenium-webdriver

  • nightwatch

  • webdriver.io

  • protractor

  • cypress

  • testcafe

  • ....

  • puppeteer

https://www.npmtrends.com/nightwatch-vs-webdriverio-vs-cypress-vs-testcafe

2019 news

  • Json Wire is OBSOLTERE, now we have W3C WebDriver, W3C Recommendation 05 June 2018 http://www.w3.org/TR/webdriver/
  • ...
  • Chrome driver is now supported as npm package, npm install chromedriver --save-dev

nightwatch

  • uses custom runer ;-(
  • can use mocha as runner, comes bundled with 5.2 while latest mocha is 6.0 ;-(
  • but can be used with "standard mocha" with some boiler plate code ;-)
  • async/await is available (as beta)
  • custom commands https://nightwatchjs.org/guide#extending-nightwatch

cypress

https://crossbrowsertesting.com/blog/test-automation/selenium-vs-cypress/

webdriver.io

Compared

OCR

ui-testing 2015

Experiments and reviews of various ui testing frameworks

How this repo was setup

mkdir ui-testing
cd ui-testing

echo '{ "name":"ui-testing"}' > package.json

npm install --save-dev selenium-webdriver
npm install --save-dev browserstack-webdriver
npm install --save-dev wd
npm install --save-dev webdriverio
npm install --save-dev kommando
npm install --save-dev nightwatch
npm install --save-dev karma
npm install --save-dev leadfoot
npm install --save-dev intern

npm install --save-dev jscoverage
npm install --save-dev blanket

git remote add origin https://github.com/ainthek/ui-testing.git

echo "node_modules" > .gitignore

Basic info

Texts come from their web sites

selenium-webdriver

The official WebDriver JavaScript bindings from the Selenium project

browserstack-webdriver

BrowserStack WebDriver JavaScript bindings with keep alive support

wd

WebDriver/Selenium 2 node.js client

webdriverio

Selenium 2.0 bindings for NodeJS

Most of the Selenium WebDriver Wire Protocol is already implemented and wrapped in useful commands.

Adding helper functions, or more complicated sets and combinations of existing commands is simple and really useful

WebdriverIO works in combination with most of the TDD and BDD test frameworks in the JavaScript world.

nightwatch

A node.js bindings implementation for selenium 2.0/webdriver

UI automated testing framework powered by Node.js. It uses the Selenium WebDriver API.

kommando

Configurable cross browser functional / acceptance test launcher (using Webdriver)

karma

Spectacular Test Runner for JavaScript.

A simple tool that allows you to execute JavaScript code in multiple real browsers.

The main purpose of Karma is to make your TDD development easy, fast, and fun

intern

Intern. A next-generation code testing stack for JavaScript.

Intern is a complete test stack for JavaScript designed to help you write and run consistent, high-quality test cases for your JavaScript libraries and applications. It can be used to test any JavaScript code.

Its functional testing capabilities can even be used to test non-JavaScript Web and mobile apps, if you really want.

blanket

A seamless JavaScript code coverage library.

jscoverage

a javascript coverage tool, can be used in node dev, and browser side js dev

NPM Stats

generated $(date)

Included here is not comparable, some of them are full stack things some of them are modules reused by others etc...

However the numbers tell something (TODO: separate and chart)

module downloads in the last month
$( 
s=$(
npm_stats selenium-webdriver;
npm_stats browserstack-webdriver;
npm_stats webdriverio;
npm_stats wd;
npm_stats kommando;
npm_stats webdrvr;
npm_stats nightwatch;
npm_stats karma;
npm_stats leadfoot;
npm_stats intern;
npm_stats testardo;

npm_stats mocha;

npm_stats jscoverage;
npm_stats blanket;

)

echo "$s" | md-table '\;'

#echo "<div>"
# fixme: http://stackoverflow.com/questions/13808020/include-an-svg-hosted-on-github-in-markdown
# echo "$s" | awk -F";" '{print $2" "$1}' | ../data2chart/bin/chart bar # FIXME: broken layout when published on github
#echo "</div>"

)

Modules used

$( for m in $(ls -1 node_modules) do pushd node_modules/$m > /dev/null

	echo "### $m"
	echo ""
	list_node_modules | list_node_modules_filter_nested | list_node_modules_package_json  | list_node_modules_doc | cut -d"|" -f1,2,3,4
	echo ""
	
	list_node_modules | list_node_modules_filter_nested | awk-basename | sufix "\tX" > ../../$m.modules
	popd >/dev/null


done

)

Matrix

$( join-paste *.modules | md-table | md-table-header "|" | sed "3d" | sed "s;n/a;;g" | sed "s;.modules;;g" | sed "1 s;.txt;;" rm *.modules )

$( cat ./comparison.md | md-shift-header 1 )

"