-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Need volunteers for Pilot test - May 27-June 7 #162
Comments
@zcorpan Happy to test with NVDA/Chrome and NVDA/Firefox. |
@zcorpan I am happy to test any combinations |
I'm happy to help with any of those test combinations, with the qualification that I'm still resolving some issues with our license of JAWS 2019, so I may be stuck with JAWS 2018. Which versions of the AT and browsers are we using? For macOS, we should also specify the OS version, since the VoiceOver is integrated into the OS. I currently have access to machines with High Sierra and Catalina. |
@robfentress good question! Let's discuss the issue of versions in #116 |
I cannot edit the initial comment. My guess is because I'm not a member of this GitHub repo. Please let me know which combinations I should test |
@robfentress I've added you to macOS VoiceOver/Chrome and macOS VoiceOver/Safari. @jha11y I've added you to JAWS/Chrome and JAWS/Firefox. Thanks! |
@zcorpan I'm also available to test with JAWS 2020 and Firefox/Chrome (in addition to NVDA as already indicated) if it would help. |
So that means macOS 10.15.4, Chrome 81.0.4044.138, and Safari 13.1 assuming no changes before May 11? |
Scheduling update: the new runner isn't yet ready for this pilot test, so we need to move the date. Apologies for the late rescheduling. The new plan is to have the pilot test start on Wednesday May 20 and end on Tuesday May 26. |
I’d be more than happy to do NVDA or Voiceover. Please put me down for the set that has the greatest need. |
I can help with NVDA/Firefox and NVDA/Chrome |
I've updated assignees and added instructions in the first post above. |
hello: i can test with NVDA and jaws in both Firefox and chrome and i can test all the types thanks lucy |
Scheduling update: All functionality needed for pilot test isn't ready yet. We need to move the pilot test date again, unfortunately. The new plan is Wednesday May 27 to Tuesday June 2. Thanks for your patience, and apologies for rescheduling. |
Can you please let me know if the invite is already sent to join the github team? |
@shirsha the team we use (for now) is here: https://github.com/orgs/bocoup/teams/aria-at-report-testers . You have been invited to this team. You should be able to see your invitation here: https://github.com/orgs/bocoup/invitation After this is accepted, "Sign in with GitHub" in http://aria-at.w3.org/ should work. |
@zcorpan Thankyou! |
Note: A new NVDA version came out last week or so, please make sure to upgrade. See #116 (comment) |
Note: new URL to test runner that should be used for the pilot test is http://aria-at.w3.org/ If you've previously signed in, the sign in should still work with the new URL. |
@zcorpan I can test mac OS VO/ Chrome, NVDA/Chrome |
@a11ydoer I've invited you now and assigned you in the table above. Thanks! |
@shirsha Are you using the the "run as" button when you run tests? |
Can we test yet? I haven't tried because the info because the emails say the tests would be ready starting tomorrow (May 27) |
@jha11y The pilot test has not officially started as there are still changes that need to be deployed to the app. Tomorrow is the official start of the test. |
I have few questions:
|
In the CG call tomorrow we plan to walk through how to run tests. The first post in this thread also has instructions, we'll make sure they are correct tomorrow. Here are instructions for how to join the CG call. |
I believe @mcking65 wanted another test to land today before we start the pilot test. The plan is to start after the CG call today. |
@JasonAment it looks like you still need to accept the invitation. You can do so here: https://github.com/orgs/bocoup/invitation |
"Test Pilot"
After you've assigned yourself to a test plan, the test plan name in the left-most cell becomes a link. That link will start running the tests for that test plan. See "Instructions for running pilot tests" above for more details; feel free to ask if something is still unclear. |
In the CG call we talked briefly about redistributing the assignments, after combobox was removed. I haven't made any changes though. @mcking65 and @mfairchild365 can you follow up with this? I don't know what you wanted to test. This could be done now in the app interface directly, and ignore the table above, if that is easier. |
@JasonAment, hmm that's weird. I've invited you again now. |
Let's let people self-assign in the app interface. If you were assigned to the combobox test plan, or if you were not previously assigned to a test plan, or if you just have some extra time on your hands, feel free to self-assign to either the checkbox or menubar plans. We should have at least two testers for each combination of test plan + AT/browser. If you are not sure how or where to assign yourself, comment here and we can help. |
Thank you to everyone who contributed in the pilot test! I look forward to hearing about this in the CG call today. What worked well, what could be improved. I see a lot of issues filed, many from @robfentress - thank you! I haven't yet reviewed them in detail, but it looks like we have some work to do. Some tests are not completed. We can discuss whether to take the partial results or extend the pilot test. |
HELLO: UNFORTUNATELY I HAD TIME LAST WEEK BUT YOU SAID TO HOLD OFF AND I
WAS NOT ABLE TO GET TO THIS UNTILL TODAY IS THERE STILL A NEED FOR ME TO GO
IN AND TEST LUCY
lucia Greco
http://accessaces.com
follow me on twitter @accessaces
…On Wed, Jun 3, 2020 at 6:38 AM Simon Pieters ***@***.***> wrote:
*Thank you* to everyone who contributed in the pilot test!
I look forward to hearing about this in the CG call today. What worked
well, what could be improved. I see a lot of issues filed, many from
@robfentress <https://github.com/robfentress> - thank you! I haven't yet
reviewed them in detail, but it looks like we have some work to do.
Some tests are not completed. We can discuss whether to take the partial
results or extend the pilot test.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#162 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHDG5QYSOCJZG65CVFTIOY3RUZG3XANCNFSM4MORBM7A>
.
|
We discussed learnings from the pilot test in the CG call, minutes here: https://www.w3.org/2020/06/03-aria-at-minutes.html The feedback is fantastic. We will go through everything and make a summary report. We want to reach complete test results by at least one tester for every test plan for the pilot test. To that end, the last day of the pilot test is moved to Sunday June 7. Please comment if you're assigned and have tests left but are not able to complete further tests this week. Thanks again! |
@zcorpan Have completed all checkbox tests for JAWS/FF, JAWS/Chrome, NVDA/FF and NVDA/Chrome. Let me know if there are any problems with something or I can provide further feedback about the UX as a whole. |
Hi @accessaces! Thank you, I have invited you to the relevant GitHub team. Please see the top of this issue for instructions, and coordinate with @mfairchild365 and @mcking65 with assignments. |
I'll finish the menubar testing with VoiceOver using Chrome by the end of day on June 7th (probably sooner). |
I have labeled all issues today that were reported during the pilot test.
We are also working on a document with more details on challenges and learnings. |
We need 6 to 12 volunteers to run the pilot test May 27 (starting after the CG call) to June 7 (time limit was extended from June 2 to June 7).
The goal is to run through 3 test plans and produce test results, so we know that it works before we start the first complete test cycle.
If you are able to run tests (i.e., you know how to use the relevant screen reader), it would be great if you could contribute time that week for this. Please edit this comment to add your github username, or comment below. :-)
See #162 (comment) regarding assignments.
(Removed combobox, since it's not ready.)
From what I understand, @jongund @mcking65 and @mfairchild365 wanted to test everything. The runner only allows testers to assign themselves to a test plan if it currently has 0 or 1 assignees, but the admin interface allows assigning additional testers.
How to volunteer
Comment below with what you can test. (We need your GitHub username, so commenting here makes this easy.)
Instructions for running pilot tests
On the test page, there is an iframe that contains the test-specific instructions, a toolbar to navigate between tests within the test plan, and buttons:
What will happen with the pilot test results?
This is a pilot test to make sure the test runner app, test reports and the tests themselves work as anticipated. Any feedback you have is very welcome, please file an issue! The test results for this will be public but will not be published as being "official", since the tests themselves haven't yet been reviewed.
Later, we plan to do a full test cycle of all tests that are reviewed, and the results of that will be published as the first set of official test results.
The text was updated successfully, but these errors were encountered: