Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to support newest AT-Driver specification #16

Merged
merged 20 commits into from
Sep 21, 2023

Conversation

gnarf
Copy link
Contributor

@gnarf gnarf commented Sep 13, 2023

This is the work in progress with the changes I needed to make to get this working with the PAC driver which speaks the newer version of the at-driver spec.

Opening this up as a draft pull request because these changes will need to make it into the repo, but also hoping @mzgoddard might be able to give me some pointers on where to organize things/any coding conventions I've not followed yet.

Still TODO:

  • rebase onto latest main branch
  • Ensure tests still work and pass
  • Update mock drivers if needed
  • Move selinium key mappings to a better location maybe?

@gnarf gnarf requested a review from mzgoddard September 13, 2023 16:49
Copy link
Contributor Author

@gnarf gnarf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some notes for the first draft

src/agent/at-driver.js Show resolved Hide resolved
src/agent/at-driver.js Outdated Show resolved Hide resolved
src/agent/at-driver.js Outdated Show resolved Hide resolved
src/agent/at-driver.js Outdated Show resolved Hide resolved
src/agent/driver-test-runner.js Show resolved Hide resolved
src/agent/messages.js Outdated Show resolved Hide resolved
src/agent/web-driver.js Outdated Show resolved Hide resolved
@gnarf
Copy link
Contributor Author

gnarf commented Sep 13, 2023

Rebased onto cb23119

src/agent/at-driver.js Outdated Show resolved Hide resolved
Copy link
Contributor

@jugglinmike jugglinmike left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, Corey!

Beware that this project has an undocumented dependency on TypeScript. While I don't think we should worry about preserving that integration here in this patch, I'm thinking that formalization is a piece of technical debt we should pay down before this code sees production use. (On that note, it's a bit concerning that we were able to make all this change without breaking a single test, so I'm also going to think about when/how we should improve coverage.)

src/agent/at-driver.js Outdated Show resolved Hide resolved
src/agent/at-driver.js Outdated Show resolved Hide resolved
src/agent/at-driver.js Outdated Show resolved Hide resolved
src/agent/at-driver.js Outdated Show resolved Hide resolved
if (message.type === 'event' && message.name === 'speech') {
yield message.data;
if (message.method === 'interaction.capturedOutput') {
yield message.params.data;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

src/agent/messages.js Outdated Show resolved Hide resolved
src/agent/at-driver.js Outdated Show resolved Hide resolved
@gnarf
Copy link
Contributor Author

gnarf commented Sep 13, 2023

Regarding TypeScript, I haven't yet installed it on my latest machine but I'm more than happy to to continue the informal support especially if I've broken it with something in this commit. If you think I should

Copy link
Contributor

@mzgoddard mzgoddard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My high level thoughts:

  1. Uncomment and disambiguate the DEBUG messages to their own message types. Do human readable formatting in messages.js. The first goal of harness output is to be machine readable. From what I recall harness will not make final judgement if assertions "pass". It is meant to run and collect AT speech/responses. The log calls should be machine readable.
    1. The pass member that is currently there was from before we had discussions better defining responses and verdicts for automation.
    2. I'm hesitant for us to have a DEBUG message. If its something that is formatting variables it should probably be a unique message.
  2. Validate the selenium key mapping in validateKeysFromCommand. I think it'd be good for validateKeysFromCommand to return two versions of ATKey one with the original test version, and a separate one processed for sending through at-driver. If harness needs to do this translation we should log both the version written in the test, and the version sent to at-driver.
    1. I'd guess the validation would be if not in seleniumKeyMap and key.length > 1 { not valid key }

@gnarf gnarf marked this pull request as ready for review September 20, 2023 15:52
@gnarf
Copy link
Contributor Author

gnarf commented Sep 20, 2023

Not sure what I need to do to make sure I've validated for typescript as well (or if I need to). Is it just "run the tests" or do I need to install/run something different.

@gnarf
Copy link
Contributor Author

gnarf commented Sep 20, 2023

I am noticing that these "at driver comms" messages are being output to the harness output format now, in the "log" sections, and probably shouldn't be. It's basically redundant information only exposed to help someone in my position figure out what inconsistencies with the communications protocol might be causing an error "real-time".

Example "output" of running a test:

{
    "name": "unknown",
    "tests": [
        {
            "filepath": "test-01-trigger-alert-reading-nvda.collected.json",
            "log": [
                {
                    "data": {
                        "type": "startTest",
                        "date": "2023-09-20T17:31:37.448Z",
                        "id": 1,
                        "title": "trigger alert"
                    },
                    "text": "Starting test #1 'trigger alert'."
                },
                {
                    "data": {
                        "type": "openPage",
                        "date": "2023-09-20T17:31:37.448Z",
                        "url": "about:blank"
                    },
                    "text": "Open page: 'about:blank'."
                },
                {
                    "data": {
                        "type": "openPage",
                        "date": "2023-09-20T17:31:37.484Z",
                        "url": {}
                    },
                    "text": "Open page: 'http://127.0.0.1:55830/1zvugs/reference/2022-4-8_144013/alert.setFocusOnButton.html'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:37.964Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:37.965Z",
                        "spokenText": " Alert Example"
                    },
                    "text": "Speech event: ' Alert Example'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:37.966Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" document\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" document\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:37.966Z",
                        "spokenText": " document"
                    },
                    "text": "Speech event: ' document'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:39.350Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:39.350Z",
                        "spokenText": " main landmark"
                    },
                    "text": "Speech event: ' main landmark'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:40.283Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:40.283Z",
                        "spokenText": " main landmark"
                    },
                    "text": "Speech event: ' main landmark'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:40.284Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:40.284Z",
                        "spokenText": " Run Test Setup"
                    },
                    "text": "Speech event: ' Run Test Setup'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:40.284Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:40.285Z",
                        "spokenText": " button"
                    },
                    "text": "Speech event: ' button'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:42.119Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:42.120Z",
                        "spokenText": " Alert Example"
                    },
                    "text": "Speech event: ' Alert Example'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:42.999Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:42.999Z",
                        "spokenText": " button"
                    },
                    "text": "Speech event: ' button'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:31:43.000Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:31:43.000Z",
                        "spokenText": " Run Test Setup"
                    },
                    "text": "Speech event: ' Run Test Setup'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:08.322Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Trigger Alert\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Trigger Alert\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:08.322Z",
                        "spokenText": " Trigger Alert"
                    },
                    "text": "Speech event: ' Trigger Alert'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:08.324Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:08.324Z",
                        "spokenText": " button"
                    },
                    "text": "Speech event: ' button'."
                },
                {
                    "data": {
                        "type": "pressKeys",
                        "date": "2023-09-20T17:32:10.255Z",
                        "keys": {
                            "type": "sequence",
                            "sequence": [
                                {
                                    "type": "chord",
                                    "keys": [
                                        {
                                            "type": "key",
                                            "key": "space",
                                            "codePoint": ""
                                        }
                                    ]
                                }
                            ]
                        }
                    },
                    "text": "Press keys: 'space'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:10.256Z",
                        "direction": "outbound",
                        "message": "{\"id\":1,\"method\":\"interaction.pressKeys\",\"params\":{\"keys\":[\"\"]}}"
                    },
                    "text": "AT-Driver: outbound: {\"id\":1,\"method\":\"interaction.pressKeys\",\"params\":{\"keys\":[\"\"]}}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:10.572Z",
                        "direction": "inbound",
                        "message": "{\"id\":1,\"result\":null}"
                    },
                    "text": "AT-Driver: inbound: {\"id\":1,\"result\":null}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:10.572Z",
                        "direction": "inbound",
                        "message": "{\"id\":1,\"result\":null}"
                    },
                    "text": "AT-Driver: inbound: {\"id\":1,\"result\":null}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:10.572Z",
                        "direction": "inbound",
                        "message": "{\"id\":1,\"result\":null}"
                    },
                    "text": "AT-Driver: inbound: {\"id\":1,\"result\":null}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:10.662Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:10.662Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:10.662Z",
                        "spokenText": " alert"
                    },
                    "text": "Speech event: ' alert'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:10.663Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Hello\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Hello\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:10.663Z",
                        "spokenText": " Hello"
                    },
                    "text": "Speech event: ' Hello'."
                },
                {
                    "data": {
                        "type": "openPage",
                        "date": "2023-09-20T17:32:20.581Z",
                        "url": "about:blank"
                    },
                    "text": "Open page: 'about:blank'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:21.008Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" about:blank\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" about:blank\"}}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:21.009Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" about:blank\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" about:blank\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:21.009Z",
                        "spokenText": " about:blank"
                    },
                    "text": "Speech event: ' about:blank'."
                },
                {
                    "data": {
                        "type": "openPage",
                        "date": "2023-09-20T17:32:22.789Z",
                        "url": {}
                    },
                    "text": "Open page: 'http://127.0.0.1:55830/1zvugs/reference/2022-4-8_144013/alert.setFocusOnButton.html'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:23.071Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:23.071Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:23.071Z",
                        "spokenText": " Alert Example"
                    },
                    "text": "Speech event: ' Alert Example'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:23.073Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" document\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" document\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:23.073Z",
                        "spokenText": " document"
                    },
                    "text": "Speech event: ' document'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:24.454Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:24.454Z",
                        "spokenText": " main landmark"
                    },
                    "text": "Speech event: ' main landmark'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:25.198Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:25.198Z",
                        "spokenText": " main landmark"
                    },
                    "text": "Speech event: ' main landmark'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:25.199Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:25.199Z",
                        "spokenText": " Run Test Setup"
                    },
                    "text": "Speech event: ' Run Test Setup'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:25.200Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:25.200Z",
                        "spokenText": " button"
                    },
                    "text": "Speech event: ' button'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:27.309Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:27.309Z",
                        "spokenText": " Alert Example"
                    },
                    "text": "Speech event: ' Alert Example'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:28.331Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:28.331Z",
                        "spokenText": " button"
                    },
                    "text": "Speech event: ' button'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:28.332Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:28.332Z",
                        "spokenText": " Run Test Setup"
                    },
                    "text": "Speech event: ' Run Test Setup'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:53.142Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Trigger Alert\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Trigger Alert\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:53.142Z",
                        "spokenText": " Trigger Alert"
                    },
                    "text": "Speech event: ' Trigger Alert'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:53.144Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:53.144Z",
                        "spokenText": " button"
                    },
                    "text": "Speech event: ' button'."
                },
                {
                    "data": {
                        "type": "pressKeys",
                        "date": "2023-09-20T17:32:55.063Z",
                        "keys": {
                            "type": "sequence",
                            "sequence": [
                                {
                                    "type": "chord",
                                    "keys": [
                                        {
                                            "type": "key",
                                            "key": "enter",
                                            "codePoint": ""
                                        }
                                    ]
                                }
                            ]
                        }
                    },
                    "text": "Press keys: 'enter'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:55.064Z",
                        "direction": "outbound",
                        "message": "{\"id\":2,\"method\":\"interaction.pressKeys\",\"params\":{\"keys\":[\"\"]}}"
                    },
                    "text": "AT-Driver: outbound: {\"id\":2,\"method\":\"interaction.pressKeys\",\"params\":{\"keys\":[\"\"]}}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:55.372Z",
                        "direction": "inbound",
                        "message": "{\"id\":2,\"result\":null}"
                    },
                    "text": "AT-Driver: inbound: {\"id\":2,\"result\":null}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:55.372Z",
                        "direction": "inbound",
                        "message": "{\"id\":2,\"result\":null}"
                    },
                    "text": "AT-Driver: inbound: {\"id\":2,\"result\":null}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:55.372Z",
                        "direction": "inbound",
                        "message": "{\"id\":2,\"result\":null}"
                    },
                    "text": "AT-Driver: inbound: {\"id\":2,\"result\":null}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:55.456Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:55.456Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:55.456Z",
                        "spokenText": " alert"
                    },
                    "text": "Speech event: ' alert'."
                },
                {
                    "data": {
                        "type": "atDriverComms",
                        "date": "2023-09-20T17:32:55.460Z",
                        "direction": "inbound",
                        "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Hello\"}}"
                    },
                    "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Hello\"}}"
                },
                {
                    "data": {
                        "type": "speechEvent",
                        "date": "2023-09-20T17:32:55.460Z",
                        "spokenText": " Hello"
                    },
                    "text": "Speech event: ' Hello'."
                },
                {
                    "data": {
                        "type": "openPage",
                        "date": "2023-09-20T17:33:05.373Z",
                        "url": "about:blank"
                    },
                    "text": "Open page: 'about:blank'."
                }
            ],
            "results": [
                {
                    "testId": 1,
                    "commands": [
                        {
                            "command": "SPACE",
                            "output": " alert\n Hello"
                        },
                        {
                            "command": "ENTER",
                            "output": " alert\n Hello"
                        }
                    ],
                    "results": [
                        {
                            "command": "SPACE",
                            "expectation": "Role 'alert' is conveyed",
                            "pass": true
                        },
                        {
                            "command": "SPACE",
                            "expectation": "Text 'Hello' is conveyed",
                            "pass": true
                        },
                        {
                            "command": "ENTER",
                            "expectation": "Role 'alert' is conveyed",
                            "pass": true
                        },
                        {
                            "command": "ENTER",
                            "expectation": "Text 'Hello' is conveyed",
                            "pass": true
                        }
                    ]
                }
            ]
        }
    ],
    "log": [
        {
            "data": {
                "type": "startTest",
                "date": "2023-09-20T17:31:37.448Z",
                "id": 1,
                "title": "trigger alert"
            },
            "text": "Starting test #1 'trigger alert'."
        },
        {
            "data": {
                "type": "openPage",
                "date": "2023-09-20T17:31:37.448Z",
                "url": "about:blank"
            },
            "text": "Open page: 'about:blank'."
        },
        {
            "data": {
                "type": "openPage",
                "date": "2023-09-20T17:31:37.484Z",
                "url": {}
            },
            "text": "Open page: 'http://127.0.0.1:55830/1zvugs/reference/2022-4-8_144013/alert.setFocusOnButton.html'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:37.964Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:37.965Z",
                "spokenText": " Alert Example"
            },
            "text": "Speech event: ' Alert Example'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:37.966Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" document\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" document\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:37.966Z",
                "spokenText": " document"
            },
            "text": "Speech event: ' document'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:39.350Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:39.350Z",
                "spokenText": " main landmark"
            },
            "text": "Speech event: ' main landmark'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:40.283Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:40.283Z",
                "spokenText": " main landmark"
            },
            "text": "Speech event: ' main landmark'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:40.284Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:40.284Z",
                "spokenText": " Run Test Setup"
            },
            "text": "Speech event: ' Run Test Setup'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:40.284Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:40.285Z",
                "spokenText": " button"
            },
            "text": "Speech event: ' button'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:42.119Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:42.120Z",
                "spokenText": " Alert Example"
            },
            "text": "Speech event: ' Alert Example'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:42.999Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:42.999Z",
                "spokenText": " button"
            },
            "text": "Speech event: ' button'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:31:43.000Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:31:43.000Z",
                "spokenText": " Run Test Setup"
            },
            "text": "Speech event: ' Run Test Setup'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:08.322Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Trigger Alert\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Trigger Alert\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:08.322Z",
                "spokenText": " Trigger Alert"
            },
            "text": "Speech event: ' Trigger Alert'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:08.324Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:08.324Z",
                "spokenText": " button"
            },
            "text": "Speech event: ' button'."
        },
        {
            "data": {
                "type": "pressKeys",
                "date": "2023-09-20T17:32:10.255Z",
                "keys": {
                    "type": "sequence",
                    "sequence": [
                        {
                            "type": "chord",
                            "keys": [
                                {
                                    "type": "key",
                                    "key": "space",
                                    "codePoint": ""
                                }
                            ]
                        }
                    ]
                }
            },
            "text": "Press keys: 'space'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:10.256Z",
                "direction": "outbound",
                "message": "{\"id\":1,\"method\":\"interaction.pressKeys\",\"params\":{\"keys\":[\"\"]}}"
            },
            "text": "AT-Driver: outbound: {\"id\":1,\"method\":\"interaction.pressKeys\",\"params\":{\"keys\":[\"\"]}}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:10.572Z",
                "direction": "inbound",
                "message": "{\"id\":1,\"result\":null}"
            },
            "text": "AT-Driver: inbound: {\"id\":1,\"result\":null}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:10.572Z",
                "direction": "inbound",
                "message": "{\"id\":1,\"result\":null}"
            },
            "text": "AT-Driver: inbound: {\"id\":1,\"result\":null}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:10.572Z",
                "direction": "inbound",
                "message": "{\"id\":1,\"result\":null}"
            },
            "text": "AT-Driver: inbound: {\"id\":1,\"result\":null}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:10.662Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:10.662Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:10.662Z",
                "spokenText": " alert"
            },
            "text": "Speech event: ' alert'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:10.663Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Hello\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Hello\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:10.663Z",
                "spokenText": " Hello"
            },
            "text": "Speech event: ' Hello'."
        },
        {
            "data": {
                "type": "openPage",
                "date": "2023-09-20T17:32:20.581Z",
                "url": "about:blank"
            },
            "text": "Open page: 'about:blank'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:21.008Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" about:blank\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" about:blank\"}}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:21.009Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" about:blank\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" about:blank\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:21.009Z",
                "spokenText": " about:blank"
            },
            "text": "Speech event: ' about:blank'."
        },
        {
            "data": {
                "type": "openPage",
                "date": "2023-09-20T17:32:22.789Z",
                "url": {}
            },
            "text": "Open page: 'http://127.0.0.1:55830/1zvugs/reference/2022-4-8_144013/alert.setFocusOnButton.html'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:23.071Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:23.071Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:23.071Z",
                "spokenText": " Alert Example"
            },
            "text": "Speech event: ' Alert Example'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:23.073Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" document\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" document\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:23.073Z",
                "spokenText": " document"
            },
            "text": "Speech event: ' document'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:24.454Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:24.454Z",
                "spokenText": " main landmark"
            },
            "text": "Speech event: ' main landmark'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:25.198Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" main landmark\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:25.198Z",
                "spokenText": " main landmark"
            },
            "text": "Speech event: ' main landmark'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:25.199Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:25.199Z",
                "spokenText": " Run Test Setup"
            },
            "text": "Speech event: ' Run Test Setup'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:25.200Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:25.200Z",
                "spokenText": " button"
            },
            "text": "Speech event: ' button'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:27.309Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Alert Example\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:27.309Z",
                "spokenText": " Alert Example"
            },
            "text": "Speech event: ' Alert Example'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:28.331Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:28.331Z",
                "spokenText": " button"
            },
            "text": "Speech event: ' button'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:28.332Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Run Test Setup\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:28.332Z",
                "spokenText": " Run Test Setup"
            },
            "text": "Speech event: ' Run Test Setup'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:53.142Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Trigger Alert\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Trigger Alert\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:53.142Z",
                "spokenText": " Trigger Alert"
            },
            "text": "Speech event: ' Trigger Alert'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:53.144Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" button\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:53.144Z",
                "spokenText": " button"
            },
            "text": "Speech event: ' button'."
        },
        {
            "data": {
                "type": "pressKeys",
                "date": "2023-09-20T17:32:55.063Z",
                "keys": {
                    "type": "sequence",
                    "sequence": [
                        {
                            "type": "chord",
                            "keys": [
                                {
                                    "type": "key",
                                    "key": "enter",
                                    "codePoint": ""
                                }
                            ]
                        }
                    ]
                }
            },
            "text": "Press keys: 'enter'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:55.064Z",
                "direction": "outbound",
                "message": "{\"id\":2,\"method\":\"interaction.pressKeys\",\"params\":{\"keys\":[\"\"]}}"
            },
            "text": "AT-Driver: outbound: {\"id\":2,\"method\":\"interaction.pressKeys\",\"params\":{\"keys\":[\"\"]}}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:55.372Z",
                "direction": "inbound",
                "message": "{\"id\":2,\"result\":null}"
            },
            "text": "AT-Driver: inbound: {\"id\":2,\"result\":null}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:55.372Z",
                "direction": "inbound",
                "message": "{\"id\":2,\"result\":null}"
            },
            "text": "AT-Driver: inbound: {\"id\":2,\"result\":null}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:55.372Z",
                "direction": "inbound",
                "message": "{\"id\":2,\"result\":null}"
            },
            "text": "AT-Driver: inbound: {\"id\":2,\"result\":null}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:55.456Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:55.456Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" alert\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:55.456Z",
                "spokenText": " alert"
            },
            "text": "Speech event: ' alert'."
        },
        {
            "data": {
                "type": "atDriverComms",
                "date": "2023-09-20T17:32:55.460Z",
                "direction": "inbound",
                "message": "{\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Hello\"}}"
            },
            "text": "AT-Driver: inbound: {\"method\":\"interaction.capturedOutput\",\"params\":{\"sessionid\":\"e38a0e03-c5ee-44ff-9a2e-e6af3505781f\",\"data\":\" Hello\"}}"
        },
        {
            "data": {
                "type": "speechEvent",
                "date": "2023-09-20T17:32:55.460Z",
                "spokenText": " Hello"
            },
            "text": "Speech event: ' Hello'."
        },
        {
            "data": {
                "type": "openPage",
                "date": "2023-09-20T17:33:05.373Z",
                "url": "about:blank"
            },
            "text": "Open page: 'about:blank'."
        }
    ]
}

@jugglinmike
Copy link
Contributor

Not sure what I need to do to make sure I've validated for typescript as well (or if I need to). Is it just "run the tests" or do I need to install/run something different.

Thanks for calling this out, but we don't need to worry about it, now. (I don't know how to validate the types, either.)

I am noticing that these "at driver comms" messages are being output to the harness output format now, in the "log" sections, and probably shouldn't be. It's basically redundant information only exposed to help someone in my position figure out what inconsistencies with the communications protocol might be causing an error "real-time".

We're really only interested in the "results" object because that's all we need to construct a payload for ARIA-AT App. Consumers who find the logging data noisy have the ability to filter the messages by type via command-line arguments. That said, would you prefer to remove those messages?

@gnarf
Copy link
Contributor Author

gnarf commented Sep 21, 2023

We're really only interested in the "results" object because that's all we need to construct a payload for ARIA-AT App. Consumers who find the logging data noisy have the ability to filter the messages by type via command-line arguments. That said, would you prefer to remove those messages?

Even without enabling the logging (without --debug or other log messages), the log messages are still ending up in the output json "log" section. It is really easy to just ingore the "log" portions of the output, but I feel like this particular "class" of information should probably not make it into the "permanent records" of the output (it's a turn it on in case of emergency debug needs kind of option)

I don't feel a need to change it, but it's very much triggering some "signal to noise" aspect of the output for me.

@jugglinmike
Copy link
Contributor

Okay, go ahead and remove it

@gnarf gnarf changed the title [Draft] Changes to support newest AT-Driver specification Changes to support newest AT-Driver specification Sep 21, 2023
@gnarf
Copy link
Contributor Author

gnarf commented Sep 21, 2023

okay, my attempts to make it ignore these messages in the output have been unsuccessful, but... I still think it's worth seeing the output on the console in the meantime. I think we could add a "extra verbose" concept to the log messages and make it not add these messages to the eventual "test plan json output" format, or just continue ignoring them in the output json forever :)

@jugglinmike jugglinmike merged commit 9b67b40 into w3c:main Sep 21, 2023
6 checks passed
@jugglinmike jugglinmike deleted the pac-nvda branch September 21, 2023 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants