XPath element not found; Chrome extension only records as a hidden title

[Cathy21]

I'm currently writing a script to automate data insertion in a company website. I am trying to type into a input field but TagUI can not find it. The element I am trying to find is

<input type = "text" class="textfield autosuggesttextfield" autocomplete="off" tit;e="3rd Item Number" maxlength="28" tabindex="2" name="qbe0_1.0" style="width: 156px; padding-bottom: 1px; cursor: auto;" prevwidth="156">

I tried using the TagUI chrome extension but for every action it says "click menutitle_hidden"
Is this an issue with the site I am using? I can login and navigate to the specific page in TagUI just fine but it can not find any element on the page.

[kensoh]

Hi @Cathy21, yes it is likely due to the website you are automating on. The Chrome recorder is not fool-proof because of many variations in web design standards. You can try using the name attribute by type qbe0_1.0 as dummy text for field or try using below to find the XPath and use that -

https://github.com/kelaberetiv/TagUI#find-xpath-of-web-element

[Cathy21]

Thank you for the quick response! I checked the path and it was inside an iframe so I used frame{} - I can now type into the input box but I am unable to click on the checkbox. This is the checkbox - only the highlighted area corresponds to the checkbox element - anything else below is associated to other elements in the same row.

This is after I click the checkbox normally.

I am using the XPath to locate the checkbox and the name does not change.

frame e1menuAppIframe
{
type qbe0_1.0 as '+$itemno+'[enter]
click //*[@id="G0_1_R0"]/td[1]/div/input[@name="grs0_1"]
}

[kensoh]

Hi Cathy, normally below code will work for this situation, since name is usually unique identifier -

frame e1menuAppIframe
{
type qbe0_1.0 as `$itemno`[enter]
click //*[@name="grs0_1"]
}

If above does not work, it could be a non-standard design in checkbox. Try the following -

1. consider using dom step to use JavaScript and control the clicking or setting it to the way that it is clicked (if you are able to do that in the Chrome inspector console)

2. consider using visual automation to actually simulate the computer mouse clicking on it (see homepage for more details on how to use that and the pre-requisite Java)

If above cannot work, you can try sharing your replication code here so that I can take a look.

PS - the backtick is now the new way of denoting variables - eg above for $itemno
but using '+variable+' like what you did is still supported for backward compatibility.

[Cathy21]

I tried using the code that you provided but it did not work.

Here is what I tried for your suggestions:

dom document.getElementsByClassName("grs0_1").click();
dom document.getElementsByClassName("grs0_1").checked = true

I don't believe there is a way to set it so that it is clicked. The only indication that the checkbox is marked are the changes that appear in the photo from my latest reply. The two lines of code I tried did not work (tried separately).

• Visual Automation with a .png image of the checkbox does not work properly - the mouse moves to a different region of the page and not the checkbox (there is no error it just moves to a random spot on the webpage and indicates a success in the tagui_windows log). There are also multiple checkboxes located on the page. In addition the checkbox corresponds to a specific row of a table so I would need to check the row information before knowing which checkbox to select.
  -Visual Automation with coordinates seems to be the only option that is working - however, this is not an ideal solution for my program because it will be used on multiple laptops with different sized screens and I would therefore need to adjust the coordinates for each laptop individually.

tl:dr - Only Visual Automation with coordinates works but is not ideal for the process I am automating.

I can send the script and screenshots of the chrome inspector console for the page I am automating but unfortunately it is a company page that can only be accessed with specific credentials and on the company server. (I will email the script and screenshots)

Once again, thank you so much for the quick response and help! As a first time user of TagUI and automation tools in general, it has been very helpful.

[kensoh]

Hi @Cathy21 got it, I received the details in your email. It looks like after clicking, the only change is the class of the checkbox has an additional selectedModifier. Considering all the inputs so far, the next suggestion would also rely on the visual automation engine.

1. first try setting your Windows laptop display resolution to recommended or 100%. One of them would make the visual automation capability work. The behaviour you mentioned above clicking at the wrong spot happens when the engine interpreted the screen resolution wrongly.

2. After that, hopefully for clicking the checkbox you described above, is it possible to be done completely using the keyboard? Eg ctrl+f to search for the correct row and tab backward and space to click? If yes, you can use the keyboard step to replicate the complex keystrokes to make that selection. Eg keyboard [ctrl]f, keyboard `string` etc etc. For the syntax see here - New keyboard step for advanced keyboard entry and modifiers #370

[Cathy21]

When I tried the visual automation in my previous response, I tried it on both the recommended resolution and 100% but it was still clicking at the wrong spot- I currently have an external monitor attached that is extended from my laptop but I took the screenshot on the laptop screen which is where the automation runs.

It seems like the program is having a lot of difficulty finding many of the elements in the frame in general. For now, I am using keystrokes to navigate through the frame I am working in and that works pretty well.

I tried the Ctrl f option and that works pretty well on a checkbox on another page I am automating and therefore no longer need the mouse coordinates! Unfortunately the page with the checkbox I was originally specifying has a weird tabbing order and would require me to tab through almost half the elements on the page before I get to the checkbox. However the easy workaround for this specific site was just to customize the columns that appear so that I could tab directly from the search bar.

Overall, the general solution was using vision type(Key.TAB) to navigate and vision type(Key.SPACE) to click and customizing components of the website to reduce the time it takes to tab to the correct element.

Thanks for the help! While this may not be an ideal solution for everyone, it works well enough on the website I am using to get the job done!

[kensoh]

Great, thanks for sharing the update!

You can try below to see if that speeds up, by bundling keystrokes in 1 command -

keyboard [tab][tab][tab][tab][tab][tab][tab][tab][tab]

[Cathy21]

Is there a difference between keyboard and vision type? Is there one that is preferred over the other?

[kensoh]

Yes, vision is a workaround in the past to send custom commands to SikuliX, the computer vision and OCR engine used by TagUI. But since TagUI v5, keyboard step is introduced and that is the preferred (easier and more readable) way to send complex keystrokes. Both ways work.

TagUI v4

vision type(Key.TAB)
vision type(Key.TAB, Key.ALT)

TagUI v5

keyboard [tab]
keyboard [alt][tab]

[Cathy21]

Using keyboard and bundling keystrokes was much faster! Thank you for all the help :)

[Cathy21]

Hi @kensoh
I just had a question regarding my initial question for typing into the input box. My script iterates through an array and inputs the item number into that input box using type qbe0_1.0 as dummy text for field. It seems like the script will type correctly for the first iteration but once it goes to the second iteration, it no longer types into that same input box. Does this have something to do with it being in the frame? Should put my for loop inside or outside the frame{} section?
My current structure is:

frame frame_name
{
for (n=0;n<some_length;n++)
{
type qbe0_1.0 as `array[n]`
...
}
}

[kensoh]

I haven't come across this issue of loops in frame. Your structure looks correct.

First check if the qbe0_1.0 identifier change with each iteration?

If the identifier does not change, try the following to see if it helps -

for (n=0;n<some_length;n++)
{
frame frame_name
{
type qbe0_1.0 as `array[n]`
...
}
}

[kensoh]

Closing issue for now until further inputs!