Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add user test 1 resources! #26

Merged
merged 6 commits into from
Nov 10, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions user-tests/0-support/generic-recording-consent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Recording consent form

Thank you for participating in our usability research on Space Telescope Science Institute public notebooks.

We will be recording your session to allow Space Telescope Science Institute and external grant team members who are unable to be here today to observe your session and benefit from your comments. Please read the statement below and sign where indicated.

- I understand that my usability test session will be recorded.
- I understand that my usability test session recording will be stored on private Space Telescope Science Institute drives.
- I understand that anonymized summaries of information learned from my session may be publicly shared on the grant repository.
- I grant Space Telescope Science Institute permission to use this recording for internal use only, for the purpose of improving the designs being tested.

Signature:

Print name:

Date:

## Notes

Adapted from [Steve Krug's Recording consent form](https://sensible.com/download-files/).

This document was further adapted by the institutions handling all paperwork for participants to meet their own needs. This example is here as a reference for how we collaborated with other teams.
34 changes: 34 additions & 0 deletions user-tests/0-support/outreach-email-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# User testing outreach email templates

Subject line: Paid Notebook Accessibility Feedback Session

## Version 2

Hi {{name}},

I’m {{sender}}, and I’m reaching out on behalf of the Space Telescope Science Institute and the Project Jupyter community. We’d like to invite you to join us as a paid participant in our upcoming user tests to learn more about Jupyter notebooks and their accessibility needs in {{month}}. We got your contact information from {{referral}}.

Our team has been working on improving the inclusivity of our large collection of notebooks and the notebooks Space Telescope scientists will be making in the future. One of our main goals is to evaluate how easy it is to navigate and understand our public non-editable notebooks, and we want your feedback. During the one-hour feedback sessions we’d be asking you to complete some tasks while navigating a preset notebook. We can offer $100 per hour-long session. If you are interested, let me know and I will follow up with scheduling information.

Feel free to reach out if you have any other questions. If you are interested in joining us and can’t make that timeline, please let me know. We are planning multiple rounds of user testing and may be able to schedule one that works for you.

Best,
{{sender}}

## Version 2

Hi {{name}},

This is {{sender}} from {{affiliation}}. It’s been awhile since we’ve spoken, I hope you’re doing well!

Are you interested in being a paid participant in upcoming user tests about Jupyter notebooks and their accessibility. We can offer $100 per hour-long session. We are aiming to schedule the first round of tests roughly between {{date 1}} and {{date 2}}.

If you are interested, please pick a time using this calendy link:

If you have trouble with calendy, here’s a link to some accessibility help:
or feel free to respond and tell us what date/times work best for you and we can schedule that way.

Please email me with any other questions. If you are interested in joining us and can’t make that timeline, let me know—we are planning multiple rounds of user testing and may be able to schedule one that works for you.

Best,
{{sender}}
29 changes: 29 additions & 0 deletions user-tests/0-support/possible-test-notebooks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Possible test notebooks

A list of existing notebooks that could be used during our rounds of user tests. They may also provide inspiration to custom test notebooks we may make.

| Notebook or collection name | Test candidate? | Used in test? (date) |
| --- | --- | --- |
| [JupyterLab examples test.ipynb](https://github.com/jupyterlab/jupyterlab/blob/master/examples/notebook/test.ipynb) | | |
| [nbconvert examples](https://github.com/jupyter/nbconvert-examples) | | |
| [Lorenz notebook](https://github.com/jupyterlab/jupyterlab-demo/blob/master/notebooks/Lorenz.ipynb) | Yes | |
| [JupyterLab benchmarks test noteboks](https://github.com/jupyterlab/benchmarks/tree/master/examples/from-benchmarks) | | |
| [Cosmic Origins Spectograph notebooks](https://www.stsci.edu/hst/instrumentation/cos/documentation/notebooks) | | |
| [STScI notebook style guide template](https://github.com/spacetelescope/style-guides/blob/master/templates/example_notebook.ipynb) | | Yes (Test 1, August 2022) |
| [STScI notebook style guide](https://github.com/spacetelescope/style-guides/blob/master/guides/jupyter-notebooks.md) | Yes | |
| [STScI JDAT notebooks](https://github.com/spacetelescope/jdat_notebooks/tree/main/notebooks) [(Rendered here)](https://spacetelescope.github.io/jdat_notebooks/intro.html) | | |
| [STScI JWebbinar notebooks](https://github.com/spacetelescope/jwebbinar_prep) | Maybe (needs content review) | |


The following are axes we considered choosing notebooks based on. If they end up being a deciding factor we take note of, then they will be added to the above table.

- length (in cells)
- has MD cells
- has code cells
- has text-only outputs
- has non-text outputs
- notes about outputs
- uses STScI data
- follows STScI notebook guidelines
- renders successfully (in viewer tool)
- Test hosts can explain it
32 changes: 32 additions & 0 deletions user-tests/0-support/user-testing-resources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# User testing resources for notebooks

Astronomy Notebooks for All - STScI

## General user testing resources

[Steve Krug usability testing guides](https://sensible.com/download-files/)
- Usability test script
- Recording consent form
- Usability Testing Checklists
- “Things a therapist would say”

[Remote Usability Testing: Study Guide - Nielsen Norman Group](https://www.nngroup.com/articles/remote-usability-testing-study-guide/)

[How you can perform cheap unmoderated usability testing using Zoom - UX Collective](https://uxdesign.cc/how-you-can-perform-cheap-unmoderated-usability-testing-using-zoom-9dbef023570e)

## Test-specific resources

- [jupyter/nbformat](https://github.com/jupyter/nbformat)
- [jupyter/nbconvert](https://github.com/jupyter/nbconvert)
- [jupyter/nbviewer](https://github.com/jupyter/nbviewer)
- What is the STScI equivalent for this?
- [jupyter/nbconvert-examples](https://github.com/jupyter/nbconvert-examples)
- [JupyterLab UI overview - Adobe Experience League](https://experienceleague.adobe.com/docs/experience-platform/data-science-workspace/jupyterlab/overview.html?lang=en#code-cells)
- [Some jupyterlab/jupyterlab user stories](https://github.com/jupyterlab/jupyterlab/tree/master/design)
- [The Lorenz Notebook](https://github.com/jupyterlab/jupyterlab-demo/blob/master/notebooks/Lorenz.ipynb)

## Accessibility

- [Browsing with assistive technology videos](https://tetralogical.com/blog/2021/12/24/browsing-with-assistive-technology-videos/)
- [Remotely Co-Designing Features for Communication Applications using Automatic Captioning with Deaf and Hearing Pairs](https://dl.acm.org/doi/fullHtml/10.1145/3491102.3501843)
- [CAIR Lab - RIT](http://cair.rit.edu/projects.html)
27 changes: 27 additions & 0 deletions user-tests/1-navigation/results.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Results: Structure & Navigation in Rendered Notebooks

These results are from user interviews conducted in August 2022 with [the navigation test script](test-script.md) on [the STScI tutorial sample notebook]().

## What we tested

Operating systems:
Browsers:
Assistive tech:
[Notebook (hosted at via GitHub pages)] ()
Sample size:
Method:


## How users navigated

### By headings

### By preset keys

### By zooming and skimming

### With `find` controls

## Major takeaways

## What we would do differently next time
Loading