Skip to content

Commit

Permalink
#31 #32 #24 - ask() in Chrome browser if available
Browse files Browse the repository at this point in the history
#31 - ask() shows the question in terminal window. This is not intuitive if automation is running on the Chrome browser and a prompt appears in terminal window to ask for user input. Raising an issue to update behaviour to show prompt in Chrome browser, if it's available. PS - similar issue exists if chrome_browser = False and visual_automation = True. The terminal may not be in the foreground during the automation flow to be visible for user inputs. However, there isn't a consistent way to show SikuliX popup with focus across platforms. Thus KIV for now.

#32 - At present, if visual automation mode is not on and `snap('page.png', 'screenshot.png')` is used, there is no error message saying that `init(visual_automation = True)` is needed to use this. Raising an issue to show error message instead of hanging waiting for SikuliX response.

#24 - readme linking to RedMart online groceries example to automate re-adding all items to cart (to reschedule delivery, need to cancel and create new order manually)
  • Loading branch information
kensoh committed Jul 12, 2019
1 parent 4198e0e commit 28b253a
Show file tree
Hide file tree
Showing 4 changed files with 37 additions and 22 deletions.
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,13 @@ t.close()

# API Reference

[See sample Python script](https://github.com/tebelorg/TagUI-Python/blob/master/sample.py). For web automation, web element identifier can be XPath selector, CSS selector or the following attributes - id, name, class, title, aria-label, text(), href (in decreasing order of priority). There is automatic waiting for an element to appear before timeout happens, and error is returned that the element cannot be found. To change the default timeout of 10 seconds, use timeout() function.
Check out [sample Python script](https://github.com/tebelorg/TagUI-Python/blob/master/sample.py) and [RedMart groceries example](https://github.com/tebelorg/TagUI-Python/issues/24).

An element identifier can also be a .png or .bmp image snapshot representing the UI element (can be on desktop applications or web browser). x, y coordinates of elements on the screen work as well. Transparency (0% opacity) is supported in .png images, for eg using an image of an UI element with transparent background to enable clicking on an UI element that appears on different backgrounds on different occasions.
- For web automation, web element identifier can be XPath selector, CSS selector or the following attributes - id, name, class, title, aria-label, text(), href (in decreasing order of priority). There is automatic waiting for an element to appear before timeout happens, and error is returned that the element cannot be found. To change the default timeout of 10 seconds, use timeout() function.

A further example is an image of the window or frame (PDF viewer, MS Word, textbox etc) with the center content of the image set as transparent. This allows using read() and snap() to perform OCR and save snapshots for application windows, containers, frames, textboxes with varying content. Also for read() and snap(), x1, y1, x2, y2 coordinates pair can be used to define the region of interest on the screen to perform OCR or capture snapshot.
- An element identifier can also be a .png or .bmp image snapshot representing the UI element (can be on desktop applications or web browser). x, y coordinates of elements on the screen work as well. Transparency (0% opacity) is supported in .png images, for eg using an image of an UI element with transparent background to enable clicking on an UI element that appears on different backgrounds on different occasions.

- A further example is an image of the window or frame (PDF viewer, MS Word, textbox etc) with the center content of the image set as transparent. This allows using read() and snap() to perform OCR and save snapshots of application windows, containers, frames, textboxes with varying content. Also for read() and snap(), x1, y1, x2, y2 coordinates pair can be used to define the region of interest on the screen to perform OCR or capture snapshot.

#### CORE FUNCTIONS
Function|Parameters|Purpose
Expand All @@ -99,7 +101,7 @@ send()|tagui_instruction|send TagUI instruction to TagUI for execution
#### BASIC FUNCTIONS
Function|Parameters|Purpose
:-------|:---------|:------
url()|webpage_url (blank to return current URL)|go to web URL
url()|webpage_url (no parameter to return current URL)|go to web URL
click()|element_identifier (or x, y using visual automation)| left-click on element
rclick()|element_identifier (or x, y using visual automation)|right-click on element
dclick()|element_identifier (or x, y using visual automation)|double-click on element
Expand All @@ -111,7 +113,7 @@ snap()|element_identifier (page = web page), filename_to_save|save screenshot to
load()|filename_to_load|load & return file content
dump()|text_to_dump, filename_to_save|save text to file
write()|text_to_write, filename_to_save|append text to file
echo()|text_to_echo|print text to output
echo()|text_to_echo|print text to screen
ask()|text_to_prompt|ask & return user input

#### PRO FUNCTIONS
Expand All @@ -128,9 +130,9 @@ unzip()|file_to_unzip, unzip_location (optional)|unzip zip file to specified loc
run()|command_to_run (; between commands)|run OS command & return output
dom()|statement_to_run (JavaScript code)|run code in DOM & return output
vision()|command_to_run (Python code)|run custom SikuliX commands
timeout()|timeout_in_seconds (blank to return current timeout)|change wait timeout (default 10s)
timeout()|timeout_in_seconds (blank returns current timeout)|change wait timeout (default 10s)

**keyboard() modifiers and special keys**
**keyboard() - modifiers and special keys**
>[shift] [ctrl] [alt] [cmd] [win] [meta] [clear] [space] [enter] [backspace] [tab] [esc] [up] [down] [left] [right] [pageup] [pagedown] [delete] [home] [end] [insert] [f1] .. [f15] [printscreen] [scrolllock] [pause] [capslock] [numlock]
#### HELPER FUNCTIONS
Expand Down
3 changes: 1 addition & 2 deletions sample.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Sample script to search github on Yahoo, capture screenshot of results and visit DuckDuckgo
# TagUI for Python's simple, expressive and powerful API makes digital automation fun and easy!
# pip install tagui to install TagUI for Python, pip install tagui --upgrade for latest version

Expand Down Expand Up @@ -51,12 +52,10 @@
# also supports visual element identifier using .png or .bmp image snapshot
# representing the UI element (can be on desktop applications or web browser)
# for eg t.click('start_menu.png'), t.type('username_box.png', 'Sonic')

# image transparency (0% opacity) is supported, ie images with empty sections
# t.read('image_preview_frame.png'), t.snap('application_window_frame.png')

# visual element identifiers can also be x, y coordinates of elements on the screen
# for eg t.click(600, 300), t.type(600, 300, 'Mario'), t.select(600, 300, 600, 400)

# another eg is limits of area of interest x1, y1, x2, y2 for read() and snap()
# t.read(200, 200, 600, 400), t.snap(200, 200, 600, 400, 'results.png')
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

setup(
name='tagui',
version='1.7.0',
version='1.8.0',
py_modules=['tagui'],
author='Ken Soh',
author_email='opensource@tebel.org',
license='Apache License 2.0',
url='https://github.com/tebelorg/TagUI-Python',
description='TagUI for Python is a Python package for digital process automation (RPA)',
long_description='TagUI for Python documentation here - https://github.com/tebelorg/TagUI-Python',
long_description='TagUI for Python homepage - https://github.com/tebelorg/TagUI-Python',
classifiers=[
'Development Status :: 4 - Beta',
'Programming Language :: Python :: 2',
Expand Down
36 changes: 25 additions & 11 deletions tagui.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""INTEGRATION ENGINE FOR TAGUI PYTHON PACKAGE ~ TEBEL.ORG"""
__author__ = 'Ken Soh <opensource@tebel.org>'
__version__ = '1.7.0'
__version__ = '1.8.0'

import subprocess
import os
Expand Down Expand Up @@ -659,7 +659,12 @@ def exist(element_identifier = None):
return False

# return True for keywords as the computer screen always exists
if element_identifier.lower() in ['page.png', 'page.bmp']: return True
if element_identifier.lower() in ['page.png', 'page.bmp']:
if _visual():
return True
else:
print('[TAGUI][ERROR] - page.png / page.bmp requires init(visual_automation = True)')
return False

# pre-emptive check for existence of specified image file for visual automation
if element_identifier.lower().endswith('.png') or element_identifier.lower().endswith('.bmp'):
Expand Down Expand Up @@ -840,7 +845,7 @@ def select(element_identifier = None, option_value = None, test_coordinate1 = No
return False

if element_identifier.lower() in ['page.png', 'page.bmp'] or option_value.lower() in ['page.png', 'page.bmp']:
print('[TAGUI][ERROR] - page.png and page.bmp invalid for select()')
print('[TAGUI][ERROR] - page.png / page.bmp identifiers invalid for select()')
return False

if test_coordinate1 is not None and test_coordinate2 is not None and \
Expand Down Expand Up @@ -986,15 +991,19 @@ def write(text_to_write = None, filename_to_save = None):
return True

def ask(text_to_prompt = ''):
if text_to_prompt == '':
space_padding = ''
else:
space_padding = ' '
if _chrome():
return dom("return prompt('" + _esq(text_to_prompt) + "')")

if _python2_env():
return raw_input(text_to_prompt + space_padding)
else:
return input(text_to_prompt + space_padding)
if text_to_prompt == '':
space_padding = ''
else:
space_padding = ' '

if _python2_env():
return raw_input(text_to_prompt + space_padding)
else:
return input(text_to_prompt + space_padding)

def keyboard(keys_and_modifiers = None):
if not _started():
Expand Down Expand Up @@ -1218,7 +1227,12 @@ def present(element_identifier = None):
return False

# return True for keywords as the computer screen is always present
if element_identifier.lower() in ['page.png', 'page.bmp']: return True
if element_identifier.lower() in ['page.png', 'page.bmp']:
if _visual():
return True
else:
print('[TAGUI][ERROR] - page.png / page.bmp requires init(visual_automation = True)')
return False

# check for existence of specified image file for visual automation
if element_identifier.lower().endswith('.png') or element_identifier.lower().endswith('.bmp'):
Expand Down

0 comments on commit 28b253a

Please sign in to comment.