Skip to content

Visual Automation Release
Compare
Choose a tag to compare
@kensoh kensoh released this 08 Apr 07:11
· 629 commits to master since this release
v5.0.0
3ddce04
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

TagUI is a command-line tool for digital process automation (RPA)

  • #371 - keyboard step for sending complex keystrokes to the screen
  • #369 - mouse step for low-level mouse down and up on the screen
  • #366 - interacting with UI elements by specifying their (x,y) coordinates
  • #362 - mouse_xy() / mouse_x() / mouse_y() helpers to click on offsets etc
  • #385 - enhancement to report option for auditing and tracking capability
  • #383 - graceful error handling for unexpected and unhandled errors
  • #387 - ability to disable generation of .log .raw .js files by default
  • #288, #311, #320 - improved Japanese, German, French languages
  • #347 - enabling use of dynamic variables in live mode
  • #289 - enable use of dynamic variable for wait step
  • #348 - enabling using datatable with test option
  • #376 - update upload step for new DevTools protocol
  • #275 - update upload step for new Chrome version
  • #295 - fix warning with computers running PHP 7

This is a major TagUI release where major improvements are made on visual automation capabilities. For example, web identifier, image of UI element, (x,y) coordinates of UI element can be used seamlessly by TagUI to identify and interact with UI elements. Besides web browser, this is useful for automating desktop applications using computer vision.

Using the Typeform example, below is now possible. Together with mouse_xy(), mouse_x(), mouse_y() helper functions, there are new use case possibilities such as identifying an UI element visually and then interacting with another element at some x,y offset away. Also, the keyboard step lets users send complex keystrokes to the screen effortlessly.

// sample TagUI step to enter user credentials on Typeform website
click login
type username as user@gmail.com

// besides web element identifiers, images of the elements can be used
click login_button.png
type username_box.png as user@gmail.com

// in addition, (x,y) coordinates of user-interface elements can be used
click (1200,200)
type (800,400) as user@gmail.com

Visual automation can be used directly from unzipping TagUI packaged installations. Pre-requisite is to have Java JDK (64-bit) v8 or later. TagUI v5.0 comes with SikuliX v1.1.4 - its OpenCV (for computer vision) is bumped from v2.5 to v3.4, its Tesseract (for OCR) is bumped from v2.x to v3.05.

Other notable changes include enhancement to the report option that enables easy auditing and tracking capability, enabling the use of dynamic variables in live mode for easier script development and testing, and the ability to turn off generating .log .raw .js files by default by putting an empty tagui_no_logging file in tagui/src folder, and also graceful exit on unknown error encountered.

Click to see more info on this release.

Assets 2
Loading