Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Inserter testID #18832

Merged
merged 2 commits into from
Dec 18, 2019
Merged

Add Inserter testID #18832

merged 2 commits into from
Dec 18, 2019

Conversation

jkmassel
Copy link
Contributor

@jkmassel jkmassel commented Nov 29, 2019

Description

Adds a testID to the inserter button on mobile – this allows it to be targeted by automated test tools without having to first localize the string for different device locales.

How has this been tested?

  • This PR adds a test case to ensure that the testID is present on the element
  • I've run unit tests locally, both yarn run test-unit:native and yarn test. Additionally, this PR's CI checks should be green.
  • Tested this PR in the gutenberg-mobile project, using this branch as the submodule reference, referenced in the screenshot below.

Screenshots

Screen Shot 2019-11-29 at 2 05 56 PM

Types of changes

  • Small testability improvement on mobile

Checklist:

  • My code is tested.
  • My code follows the WordPress code style.
  • My code follows the accessibility standards.
  • My code has proper inline documentation.
  • I've included developer documentation if appropriate.
  • I've updated all React Native files affected by any refactorings/renamings in this PR. .

@hypest
Copy link
Contributor

hypest commented Dec 2, 2019

Thanks for the PR @jkmassel !

I understand that adding a new prop makes searching for the specific element a bit easier, but it also feels like introducing a second way for accessing elements from tests (the first being via the accessibility labels). Not sure we'd want to have to maintain two ways like that 🤔, especially since the second one provides a shortcut to people to miss properly setting the a11y labels.

Instead, can we handle this on the testsuite side, where we could set the test helpers up such that to use the localized strings? @Tug can probably help with finding the proper functions to use the localized strings on the fly. Granted, it might not be trivial to go that way, but I'm thinking it might be more maintainable?

@jkmassel jkmassel marked this pull request as ready for review December 3, 2019 23:35
@jkmassel
Copy link
Contributor Author

jkmassel commented Dec 3, 2019

Thanks for the thoughts @hypest!

a second way for accessing elements from tests

This is sort of correct. In the past, we've relied on accessibility labels to retrieve strings, but this isn't a best practice, it's sort of a hack. Accessibility labels are meant to be for users that need accessibility tools to navigate the UI, whereas testIDs are meant to be used by tests. There are a few reasons that accessibility labels make poor identifiers:

  • They may be dynamic (as a for instance, it would be beneficial to a user to have buttons that say "delete image block", "delete text block", etc, but UI tests would have to then create a switch statement to handle this rather than just saying "find the text block", then "press delete-block-button").

  • They can change out-of-sync with the rest of the component (as a for-instance, if a label is updated, it may be some time before the localization is complete, and during that time it wouldn't be possible to use a submodule reference to use this code in a multi-language test case as the changes would be out of sync. Using an immutable identifier fixes this issue, as the developer knows the identifier ahead of time, and any changes would be made in sync).

  • It's the recommended approach (at least on iOS, I'm not entirely sure about Android) for accomplishing this task – we stick with it for all of our native UI testing, and it'd be ideal to keep Gutenberg mobile as close to the platform conventions as possible IMHO. From the linked page:

    An identifier can be used to uniquely identify an element in the scripts you write using the UI Automation interfaces. Using an identifier allows you to avoid inappropriately setting or accessing an element’s accessibility label.


In terms of your concern about maintainability, I totally understand your position – we don’t want a crazy proliferation of these all over the place that adds a maintenance burden. Fortunately, that's not necessary – many UI elements can be contextually identified already – I don't see a need to add testID in very many places. This one is necessary because the toolbar buttons are contextually ambiguous (especially as they're rearranged in RTL locales), but as of now I don't know anywhere else this would be needed.

I've added a test case in order to try to lower the maintenance burden and avoid any accidental breakage of this component, and I think that should be required any time a testID is added. Additionally, as an integration test, I'd like to add UI Tests to gutenberg-mobile that'll ensure that any testIDs in the main gutenberg project are present on the native elements when running in a simulator/emulator. This should provide sufficient test coverage to avoid any potential issues, IMHO?

Lastly, you mentioned this:

the second one provides a shortcut to people to miss properly setting the a11y labels

While that might happen, I think that the team should continue as they have to this point – testIDs are really only needed for specific cases where tests can't be built any other way, so developers won't need to add them unless they're running into difficulties with tests. testID and accessibilityLabel / hint are differentiated enough in description that I don't think someone will use the incorrect one. If we find that's becoming an issue, we could always add a linter rule to ensure that nobody is passing a localized string as the testID prop.

WDYT?

@etoledom
Copy link
Contributor

etoledom commented Dec 9, 2019

Is it the idea for this to be used in WPiOS UI tests?

What I remember from talking with @JavonDavis , is that we are not using testID because it's not recognised (found?) by the Android UI tests, so we had to fall back forcefully to accessibility labels.

I haven't tested this personally, it might be good to give it a chance once again to be sure (things might have changed since the last time this was tested). If this is still the case, we won't be able to add e2e tests in this project to ensure the existence of testIDs since they will fail in Android.

@jkmassel
Copy link
Contributor Author

jkmassel commented Dec 9, 2019

It can be used in both :)

The testID is accessible via getTag when using Espresso. This approach isn't always compatible with libraries like Appium, but it'll work fine for all of our native UI testing :)

Screen Shot 2019-12-09 at 10 37 41 AM

@etoledom
Copy link
Contributor

I see, so we will use them on both WPiOS and WPAndroid native UI Tests.

Since it's the best practice, I'd say it's good to add them, but trying to use the testID with our Appium test will probably fail (that was what wasn't working for us on Android).

I think it would be good to add a small comment about this, saying that the ID is meant to be used on native tests only, to avoid confusion and not have someone trying it and failing in Appium tests.

I'd also say that in "normal gutenberg-mobile" development, we shouldn't be expected to add testIDs, but they should be added on demand when needed by Native tests (like now, as you also mentioned 👍 )

Does it sound good? :)

@jkmassel
Copy link
Contributor Author

trying to use the testID with our Appium test will probably fail

Yes, this seems to be a current limitation of Appium.

I think it would be good to add a small comment about this

I'm definitely happy to do this, but where do you figure it would live? IMHO inline with the prop makes sense, but would we leave the message any time it's used?

I'd also say that in "normal gutenberg-mobile" development, we shouldn't be expected to add testIDs, but they should be added on demand when needed by Native tests

💯

Thanks for looking at this @etoledom!

@etoledom etoledom self-requested a review December 12, 2019 09:13
@etoledom etoledom added the Mobile App - i.e. Android or iOS Native mobile impl of the block editor. (Note: used in scripts, ping mobile folks to change) label Dec 12, 2019
@etoledom
Copy link
Contributor

etoledom commented Dec 12, 2019

where do you figure it would live? IMHO inline with the prop makes sense, but would we leave the message any time it's used?

🤔 - Inline makes sense to me too, thinking that it shouldn't be used too often. It will help since we (or I at least) usually look at these previous implementation to copy-paste inspire new components.

If we start using them all over the place, we can re-think the where to put the comment 👍

@jkmassel
Copy link
Contributor Author

@etoledom – great, thanks! I think this is ready for another review! :)

@etoledom
Copy link
Contributor

Thank you for the update @jkmassel !

Published a PR to test this change on gutenberg-mobile CI.
Let's :shipit: when wordpress-mobile/gutenberg-mobile#1681 gets ✅

Copy link
Contributor

@etoledom etoledom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI on wordpress-mobile/gutenberg-mobile#1681 is ✅ so merging now.
Thank you @jkmassel 🙏

@etoledom etoledom merged commit 3a9c5f6 into WordPress:master Dec 18, 2019
@jkmassel jkmassel deleted the add/testIDs branch December 18, 2019 18:37
@youknowriad youknowriad added this to the Gutenberg 7.2 milestone Jan 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Mobile App - i.e. Android or iOS Native mobile impl of the block editor. (Note: used in scripts, ping mobile folks to change)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants