Skip to content

Latest commit

 

History

History
574 lines (410 loc) · 49.3 KB

README.md

File metadata and controls

574 lines (410 loc) · 49.3 KB
Raw

Run Playwright tests via docker

Description

The demo at /demos/docker shows how to run Playwright tests in Docker. This is especially useful to eliminate screenshot differences when running the tests across different Operating Systems. For instance, if you develop in Windows but your CI runs on Linux.

Important

When running Playwright tests in different Operating Systems, if you are doing Visual Regression Tests, then the visual comparisons might fail. Usually the failures will be about differences in how different Operating Systems render fonts, even if it's the same font.

From this GitHub issue comment:

"for the ubuntu vs windows: font rendering & font stack resolution is handled by OS, not specced & fundamentally different, so the common agreement as of today is that there's no way to achieve consistent rendering across operating systems. The only way is to use a well-defined consistent execution environment, e.g. VM / containers"

Alternatively to finding a way to run your tests in the same Operating System as your CI, you could choose to set the treshold configuration option. Although there's a bit of configuration to do to run the Playwright tests in Docker, I think that's a preferable approach to adding a treshold because a treshold can make a visual comparison test pass when it shouldn't and lead to false positive tests. Besides treshold, configuring maxDiffPixels and maxDiffPixelRatio might help reduce the false positives.

How to build, run the app, run tests and view the test results

Important

Required dependencies:

  • Node. Tested working with v20.10.0. If you need to have different versions of node installed it's recommended that you use Node Version Manager to install and swap between node versions.
  • npm@latest: package manager used on the demos. Tested working on 10.2.5.
  • Git LFS: this demo takes screenshots and those images are uploaded to the repo using Git LFS. Without Git LFS you won't get any images when cloning the repo and the Playwright tests will fail. Tested working with 3.3.0.
  • latest Powershell: tested working on 7.4.0.
  • Docker
  • VS Code is recommended as a code editor but you can use whatever you prefer.
  1. Clone the repo.
  2. Using your favorite shell go to /demos/docker.
  3. Install the required npm packages with: 
    npm install
  4. Install the playwright browsers with: 
    npx playwright install
  5. Run the tests with: 
    npm test
    This will start a docker container which will run the app and then run the playwright tests against it.
  6. After running the tests with npm test you can view test results with: 
    npm run test:show-report
  7. Run the tests in UI mode with: 
    npm run test:ui
    This will start a docker container which will run the app and then start Playwright's UI mode. The UI mode will be served from the container and it'll be accessible at http://localhost:43008. The UI mode URL will be displayed as part of the output of the npm run test:ui command.
  8. If you just want to run the app execute the command: 
    npm start
    Once the command finishes the app should open in your default browser at http://127.0.0.1:4200/.

Run tests

What to expect

When you run npm test this is what you should expect:

docker-test.mp4

The Docker setup

The Docker setup to run Playwright tests uses the docker-compose.yml file. When npm test is executed, it runs the playwright.ps1 pwsh script which sets some environment variables before running the docker compose up command.

Available options for running tests

The npm test can be customized with the following parameters:

  • -testOptions: use this to pass any Playwright CLI supported test options to the playwright test command.

    Default value: empty string
    Example: -testOptions '--update-snapshots --grep "load page"'

  • -webServerMode: one of auto, from-docker or from-host. Determines if the Playwright tests should be executed against the Playwright Web Server running on the host or inside Docker. If you have the target test application running outside of Docker you can set this to from-host, otherwise it should be from-docker.

    Using auto will mean that the pwsh script will attempt to decide if the target test application is running on the host and if so it will use the from-host option, if not it will use the from-docker option. Setting to auto requires setting the -webServerHost and webServerPort parameters.

    Default value: auto
    Example: -webServerMode from-docker.

  • -webServerHost: the host where the Playwright's test target should be running. Required if -webServerMode auto is used.

    Default value: 127.0.0.1
    Example: -webServerHost 127.0.0.1

  • -webServerPort: the port where the Playwright's test target should be running. Required if -webServerMode auto is used.

    Default value: 4200
    Example: -webServerPort 4200

You can combine any of the above parameters. Example:

npm test '--' -testOptions '--update-snapshots --grep "load page"' -webServerMode from-docker

Note

On the example npm command above you only need the single quotes around the double dash if you're running the npm command from Powershell. For more info see Powershell and passing command line arguments to npm commands

Docker compose file

The docker-compose.yml file requires the following environment variables:

Environment variable name Description Default value Required
CI Should be set to true if running on a CI environment, false otherwise. false no
PLAYWRIGHT_VERSION Determines the Playwright's Docker image version. It must match the version of @playwright/test NPM package that is on the package.json file. --- yes
PLAYWRIGHT_TEST_OPTIONS Playwright CLI test options. --- no
NPM_INSTALL_COMMAND NPM command to install packages. Usually npm i if NOT running in a CI environment and npm ci when running on a CI environment. npm i no
USE_DOCKER_HOST_WEBSERVER Determines if the Playwright tests should be executed against the Playwright Web Server running on the host or inside Docker. If you have the target test application running outside of Docker you can set this to true, otherwise it should be false. false no

If you want to run the docker compose up command without the pwsh script then you can do so by:

  1. Go to /demos/docker.
  2. Set the required environment variables (see example below).
  3. Run docker compose up.

To set environment variables for the docker compose up command you can either create a .env file at /demos/docker and populate it. For instance:

CI=false
PLAYWRIGHT_VERSION=1.42.1
PLAYWRIGHT_TEST_OPTIONS=
NPM_INSTALL_COMMAND=npm i
USE_DOCKER_HOST_WEBSERVER=false

Or you can set them in your shell prior to running the docker compose up command. For instance, if you're using Powershell you can do:

$env:CI=false
$env:PLAYWRIGHT_VERSION=1.42.1
$env:PLAYWRIGHT_TEST_OPTIONS=
$env:NPM_INSTALL_COMMAND=npm i
$env:USE_DOCKER_HOST_WEBSERVER=false

Docker command

The docker setup to run the Playwright tests is based on the Playwright Docker docs. When running npm test, the playwright.ps1 Powershell script is executed and runs a docker compose command that is equivalent to the following docker run command:

docker run -it --rm --ipc=host `
--env CI=<true|false> `
--env USE_DOCKER_HOST_WEBSERVER=<true|false> `
--workdir=/app `
-v '<path-to-cloned-repo>\demos\docker:/app' `
-v 'npm-cache:/root/.npm' `
-v 'node-modules:/app/node_modules' `
mcr.microsoft.com/playwright:<@playwright/test-npm-package=-version>-jammy `
/bin/bash -c '<npm i | npm ci> && npx playwright test <test-options>'

Run tests with UI mode

What to expect

When the docker command to start the Playwright UI Mode completes it will display the following message:

Listening on http://0.0.0.0:<port>

Once you see this message you can access the Playwright UI by going to http://localhost:<port>.

When you run npm run test:ui this is what you should expect:

docker-test-ui.mp4

The Docker setup

The Docker setup to run Playwright tests using the UI mode uses the docker-compose.ui.yml file. When npm run test:ui is executed, it runs the playwright.ps1 pwsh script which sets some environment variables before running the docker compose -f ./docker-compose.ui.yml up command.

Available options for running tests

The npm run test:ui can be customized with the following parameters:

  • -uiPort: the port where the UI mode will be available.

    Default value: 43008
    Example: -uiPort 43008

  • -testOptions: use this to pass any Playwright CLI supported test options to the playwright test command.

    Default value: empty string
    Example: -testOptions --grep "load page"

  • -fileChangesDetectionSupportMode: one of auto, supported, or unsupported. Use this to indicate whether or not your Docker setup will be able to support file changes detection. For instance, when running Docker Desktop on Windows under WSL2 you don't have support for file changes detection but you do if you run with WSL1.

    This parameter sets the FILE_CHANGES_DETECTION_SUPPORTED environment variable which can be used by the playwright.config.ts to determine if you need to use some sort of polling mechanism for file changes instead of relying on the file system. For example, this demo runs an Angular app and if FILE_CHANGES_DETECTION_SUPPORTED is set to false then playwright.config.ts starts the target test Angular app with the --poll option so that the app is rebuild when you change the source files.

    When this is set to auto the pwsh script will set this to true if you're not running on Windows. If you're running on Windows it will set it to true if you're running on WSL1 and to false if you're running on WSL2.

    For more info see the File changes aren't triggering an application rebuild when testing with UI mode section.

    Default value: auto
    Example: -fileChangesDetectionSupportMode supported

  • -webServerMode: one of auto, from-docker or from-host. Determines if the Playwright tests should be executed against the Playwright Web Server running on the host or inside Docker. If you have the target test application running outside of Docker you can set this to from-host, otherwise it should be from-docker. Using auto will mean that the pwsh script will attempt to decide if the target test application is running on the host and if so it will use the from-host option, if not it will use the from-docker option. Setting to auto requires setting the -webServerHost and webServerPort parameters. one of auto, from-docker or from-host.

    Default value: auto
    Example: -webServerMode from-docker.

  • -webServerHost: the host where the Playwright's test target should be running. Required if -webServerMode auto is used.

    Default value: 127.0.0.1
    Example: -webServerHost 127.0.0.1

  • -webServerPort: the port where the Playwright's test target should be running.

    Default value: 4200
    Example: -webServerPort 4200

You can combine any of the above parameters. Example:

npm run test:ui '--' -testOptions '--grep "load page"' -webServerMode from-docker -fileChangesDetectionSupportMode unsupported

Note

On the example npm command above you only need the single quotes around the double dash if you're running the npm command from Powershell. For more info see Powershell and passing command line arguments to npm commands

Docker compose file

The docker-compose.ui.yml file requires the following environment variables:

Environment variable name Description Default value Required
CI Should be set to true if running on a CI environment, false otherwise. false no
PLAYWRIGHT_VERSION Determines the Playwright's Docker image version. It must match the version of @playwright/test NPM package that is on the package.json file. --- yes
PLAYWRIGHT_TEST_OPTIONS Playwright CLI test options. --- no
NPM_INSTALL_COMMAND NPM command to install packages. Usually npm i if NOT running in a CI environment and npm ci when running on a CI environment. npm i no
USE_DOCKER_HOST_WEBSERVER Determines if the Playwright tests should be executed against the Playwright Web Server running on the host or inside Docker. If you have the target test application running outside of Docker you can set this to true, otherwise it should be false. false no
UI_PORT The port for Playwright's UI Mode. When Playwright UI mode is running you'll be able to access it at http://localhost:<UI_PORT>. 43008 no
FILE_CHANGES_DETECTION_SUPPORTED Determines if you have file change detection support when running in Docker. For instance, when running Docker Desktop on Windows under WSL2 you don't have support for file changes detection but you do if you run with WSL1.

This environment can be used by the playwright.config.ts to determine if you need to use some sort of polling mechanism for file changes instead of relying on the file system. For example, this demo runs an Angular app and if FILE_CHANGES_DETECTION_SUPPORTED is set to false then playwright.config.ts starts the target test Angular app with the --poll option so that the app is rebuild when you change the source files.

For more info see the File changes aren't triggering an application rebuild when testing with UI mode section.		 false no
CHOKIDAR_USEPOLLING Similar to the FILE_CHANGES_DETECTION_SUPPORTED environment variable. This indicates if Playwright's UI Mode needs to poll for test file changes or not. Like with the FILE_CHANGES_DETECTION_SUPPORTED environment variable, this varies depending if your Docker setup supports file change detection or not. If not, then set this to true so that the UI mode updates when you make changes to the tests.

For more info see [Bug]: UI mode in Docker doesn't watch tests #29785.		 true no

If you want to run the docker compose up command without the pwsh script then you can do so by:

  1. Go to /demos/docker.
  2. Set the required environment variables (see example below).
  3. Run docker compose -f ./docker-compose.ui.yml up.

To set environment variables for the docker compose up command you can either create a .env file at /demos/docker and populate it. For instance:

CI=false
PLAYWRIGHT_VERSION=1.42.1
PLAYWRIGHT_TEST_OPTIONS=
NPM_INSTALL_COMMAND=npm i
USE_DOCKER_HOST_WEBSERVER=false
UI_PORT=43008
FILE_CHANGES_DETECTION_SUPPORTED=false
CHOKIDAR_USEPOLLING=true

Or you can set them in your shell prior to running the docker compose up command. For instance, if you're using Powershell you can do:

$env:CI=false
$env:PLAYWRIGHT_VERSION=1.42.1
$env:PLAYWRIGHT_TEST_OPTIONS=
$env:NPM_INSTALL_COMMAND=npm i
$env:USE_DOCKER_HOST_WEBSERVER=false
$env:UI_PORT=43008
$env:FILE_CHANGES_DETECTION_SUPPORTED=false
$env:CHOKIDAR_USEPOLLING=true

Docker command

The Docker setup to run the Playwright tests in UI mode is based on the Playwright Docker docs and UI Mode > Docker & GitHub Codespaces docs. When running npm test, the playwright.ps1 Powershell script is executed and runs a docker compose command that is equivalent to the following docker run command:

docker run -it --rm --ipc=host `
--env CI=<true|false> `
--env USE_DOCKER_HOST_WEBSERVER=<true|false> `
--env FILE_CHANGES_DETECTION_SUPPORTED=<true|false> `
--env CHOKIDAR_USEPOLLING=<true|false> `
--workdir=/app `
-v '<path-to-cloned-repo>\demos\docker:/app' `
-v 'npm-cache:/root/.npm' `
-v 'node-modules:/app/node_modules' `
mcr.microsoft.com/playwright:<@playwright/test-npm-package=-version>-jammy `
/bin/bash -c '<npm i | npm ci> && npx playwright test --ui-port=<ui-port> --ui-host=0.0.0.0 <test-options>'

Playwright configuration

The majority of the content of the playwright.config.ts file is what you get by default after adding Playwright to your project with npm init playwright@latest.

The main changes are:

  1. Declared a few variables at the start that are reused throughout the playwright configuration.
  2. Updated the reporter array. In addition to using the default html reporter, we've added the built-in list reporter.
  3. Defined a baseURL so that we can use relative URLs when doing page navigations on the tests.
  4. Configured the webServer block to run the Angular app locally so that the tests can be executed against it. If you're not testing an Angular app that's fine, you just need to adjust the webServer.command so that it launches your app and set the webServer.url to the url your app will be running at. For more information see the webServer docs.
  5. Defined the snapshotPathTemplate option to group snapshots by Operating System. This is a choice for this demo, it's not mandatory. This options is configured so that all snapshots generated on Unix will be stored at /tests/__screenshots__/linux and all snapshots generated on Windows will be stored at /tests/__screenshots__/win32. One of the reasons this is done is so that we can add the windows directory to the .gitignore to avoid committing windows snapshots in case someone runs the tests outside of Docker for any reason. Remember that the whole point of running in Docker is to generate Unix like snapshots to get consistent behavior between running locally and on CI, so you should never want to commit windows generated snapshots.

Note

The _isRunningOnCI variable used on the playwright.config.ts changes the value of some options when running tests on CI. To set the _isRunningOnCI variable to true you must set the environment variable CI to true before running the tests. For more information regarding using Playwright on a CI environment see Playwright docs on Continuous Integration.

Furthermore, we have created:

Note

You don't have to create the playwright.cli-options.ts or the playwright.env-vars.ts file. You can have all of this on the playwright.config.ts. Code structure is up to you.

Note

Depending on your playwright.config.ts, make sure you update your .gitignore to exclude any directory used by test results, report results, etc. Scroll to the end of this demo's .gitignore to see an example.

Other notes on the docker integration

File changes aren't triggering an application rebuild when testing with UI mode

Note

Almost sure you can ignore this section if you're not using a Windows OS.

Important

This demo solves this issue by allowing you to set the --poll option of the Angular CLI ng serve command via an extra parameter that can be passed into the npm run test:ui npm script. Check the fileChangesDetectionSupportMode option on the Run tests with UI mode > Available options for running tests section.

When running tests in UI mode you always want the app to rebuild when you change the code so that you can run tests on your latest changes.

However, there might be cases where the app doesn't rebuild when you change the code. For instance, if you're running Docker Desktop using WSL2 then you might not get file change support. As an example, for Angular apps the default watch mode will not work.

The root cause for this is explained by:

In cases where Angular CLI is running in a shared directory on linux VM on a windows host the webpack dev server isn't detecting file changes from the host environment. (ex: Whenever a docker dev environment is running on a windows host.) This is solved by adding poll option to the webpack dev server configuration. By default when no poll interval is set watchpack will use native file system change detection. This change allows for setting the poll interval which will turn on polling. 1

Although the comment above comes from an issue in the Angular CLI, the described root cause can apply to apps running on other technologies such as React, Vue, dotnet, etc.

As a workaround to forcing file changes polling, which for Angular can be done with the --poll option, you can also:

For some reason nodemon and webpack-dev-server hot reload does not work in WSL2. Downgrading to WSL 1 resolved the issue.

In order for this to work in WSL 2, the project needs to be inside the linux file system. 2

The last alternative I can provide to this issue is to run the target test app outside of Docker and then use the -webServerMode from-host option in combination with -webServerMode and -webServerPort. Example:

npm run test:ui '--' -webServerMode from-host -webServerHost 127.0.0.1 -webServerPort 4200

Since the app is running outside of Docker the file change detection will work as usual and the -webServerMode from-host will make the tests running on Docker execute against the instance of the app running on the host. For more information on the -webServerMode from-host see the Why should I use the webServerMode input parameter of the Powershell scripts ? section.

Careful when installing node modules on the docker container

This demo creates a named Docker volume for the node_modules folder and always does an install when starting the Docker containers to make sure the node_modules are up to date with the package.json.

However, if you don't want to create a volume for the node_modules and you want to mount them from the host into the Docker container then you should be careful if you use a Windows OS and your project depends on NPM packages which install different binaries depending on the OS.

Otherwise, you will get an error when the Playwright's WebServer tries to start the app inside the docker container. The error will tell you that your app failed to start using the node_modules that you mounted into the container because at least one of the NPM packages contains a binary that was built for another platform.

Here's an example of the error message:

[WebServer] An unhandled exception occurred: You installed esbuild for another platform than the one you're currently using. This won't work because esbuild is written with native code and needs to install a platform-specific binary executable.

Specifically the "@esbuild/win32-x64" package is present but this platform needs the "@esbuild/linux-x64" package instead. People often get into this situation by installing esbuild on Windows or macOS and copying "node_modules" into a Docker image that runs Linux, or by copying "node_modules" between Windows and WSL environments.

If you are installing with npm, you can try not copying the "node_modules" directory when you copy the files over, and running "npm ci" or "npm install" on the destination platform after the copy. Or you could consider using yarn instead of npm which has built-in support for installing a package on multiple platforms simultaneously.

If you are installing with yarn, you can try listing both this platform and the other platform in your ".yarnrc.yml" file using the "supportedArchitectures" feature: https://yarnpkg.com/configuration/yarnrc/#supportedArchitectures Keep in mind that this means multiple copies of esbuild will be present.

Another alternative is to use the "esbuild-wasm" package instead, which works the same way on all platforms. But it comes with a heavy performance cost and can sometimes be 10x slower than the "esbuild" package, so you may also not want to do that.

Note

The Angular app used for this demo depends on the esbuild npm package which installs Operating System specific binaries. If you tried to mount the node_modules for this demo from a Windows host into the Docker container you'd get an error like the one above.

Warning

This code demo doesn't have to deal with private NPM registries, but if you are using private registries and you need to install the NPM packages in the container, then you will have to extend this demo to pass the NPM authentication to the docker container.

For more info see:

How does the webServerMode input parameter of the Powershell scripts work ?

If you set webServerMode to from-host with npm test '--' -webServerMode from-host or with npm run test:ui '--' -webServerMode from-host then the docker container won't have to build and run your app and instead will try to connect to the app running on the host machine.

This is due to the webServer.reuseExistingServer option of the playwright.config.ts. When this is set to true then Playwright will check if the app is running on the webServer.url address and if it is then it just uses that as the target of the tests.

The trick to make this work though is that the webServer.url must be set to an address that is reachable from the docker container. And since what we want to do is to reach the app that is running on the host then what the playwright.config.ts does is set the host of the webServer.url to host.docker.internal. Furthermore, the docker container must run with the following extra host host.docker.internal:host-gateway.

For more info see:

Why should I use the webServerMode input parameter of the Powershell scripts ?

If you set the -webServerMode to from-host with npm test '--' -webServerMode from-docker or with npm run test:ui '--' -webServerMode from-host then the docker container won't have to build and run your app and instead will try to connect to the app running on the host machine.

This means that you can start the app once with npm start and then run the tests against it multiple times with npm test. Depending on your app, skipping the building and running of the app as well as the node modules install if needed as well, might result in a non-trivial time saving for your dev loop.

Tip

For most of the time I suggest using the npm test:ui for your dev loop. This command will also only and run the app once and then the UI mode will run the tests against it. When you do changes on the app or the tests everything is hot reloaded and you can keep developing and running the tests as you go.

Why are my Playwright tests running in Docker slow?

This shouldn't be the case. In my experience, Playwright runs quicker in the mcr.microsoft.com/playwright docker image than outside of docker, especially if you're running on Windows.

The issue that you might be facing though is something that I've encountered on a setup where I was mounting the node_modules directory into the container and that was affecting the filesystem performance. The Docker compose files used by this demo creates a named volume for the node_modules to avoid this problem occurring with the node_modules directory.

However, if you're using Windows and you're mounting a large number of files to the docker container then you might want to try using WSL1 as your Docker's engine.

For more information see microsoft/WSL [wsl2] filesystem performance is much slower than wsl1 in /mnt #4197, more specifically this comment.

Tip

If you're using Docker for Desktop switching the Docker engine can easily be done in the General settings. You can always toggle back to WSL2 at any time.

Toggling off WSL2 use on Docker for Desktop on Windows, means Docker will use Hyper-V. You might have to enabled it.

Playwright's test execution stops midway when running on Docker

If the tests' execution fails midway when running on Docker, it might be due to low memory available to the Docker container. Try increasing the memory limits set for Docker.

Tip

You should consider increasing the CPU and memory resources given to Docker to improve the Playwright's test execution speed. Especially the CPU resources because Playwright will try to parallelize the test execution according to the number of available CPUs.

Do I need Powershell to run Playwright in Docker?

No, this demo used Powershell to create a script with the logic to build the docker commands but you don't need it. Hopefully this demo explained the required building blocks and you can use them as you wish to create your docker commands.

Powershell and passing command line arguments to npm commands

You can use the -- notation to pass command line arguments to npm commands. However, if you're using Powershell and want to pass command line arguments to npm commands, then you should either use a double -- -- notation or single quotes like '--'. Example:

npm test '--' -testOptions '--update-snapshots'

Fore more info see:

Bonus: Cleanup Playwright stale screenshots

The Cleanup Playwright stale screenshots demo shows how you can detect and remove stale Playwright screenshots. For a better understanding of why you'd want to do this or details on the implementation of the cleanup process checkout the README for that demo.

This PR shows the changes required to integrate the solution from the Cleanup Playwright stale screenshots demo to a Playwright setup running in Docker.

You can end up with stale screenshots in many ways, such as by changing a test name or deleting a test. As the number of tests grows and you update your tests it could become problematic to have many stale screenshots hanging around. The video below shows how the npm run test:clean-screenshots added to this demo detects and deletes stale screenshots. The video shows the following:

  1. Create stale snapshots by commenting out a test to simulate a test deletion.
  2. Run the npm run test:clean-screenshots npm command with the dryRun flag to identify stale screenshots but NOT delete them. This is useful if you want a chance to review what was identified as a stale screenshot before deleting.
  3. Run the npm run test:clean-screenshots npm command without the dryRun flag which will identify and delete the stale screenshots.
  4. Run the npm run test:clean-screenshots npm command again to show that now there aren't any stale screenshots.
docker-cleanup-stale-screenshots.mp4

Note

The clean-stale-screenshots.ps1 pwsh script added to this demo is almost an exact copy of the one used in the Cleanup Playwright stale screenshots demo. The main difference is that the instruction to run the Playwright tests was changed to use the npm test command from this demo which will run the tests in Docker.

You can diff the /demos/docker/npm-pwsh-scripts/clean-stale-screenshots.ps1 with /demos/stale-screenshots-cleanup/npm-pwsh-scripts/clean-stale-screenshots.ps1 to easily identify the small difference.

Bonus: Visual Studio Code integration

This code demo also shows how you can improve your Visual Studio Code integration by configuring custom tasks to help you build and run both the app and the tests.

The available tasks are:

  • install packages: installs npm packages.
  • run app: builds and runs the app.
  • run tests: runs the Playwright tests in a docker container.
  • run tests ui: runs the Playwright tests in a docker container using UI mode.
  • show tests report: opens the test results report.

To get access to these tasks make sure you open the /demos/docker/ folder in Visual Studio Code.

Tip

You can run Visual Studio Code tasks through Quick Open (Ctrl+P) by typing 'task', Space and the command name. For more info see Integrate with External Tools via Tasks.

Example running the tests using the Visual Studio Code task

When you run the run tests task you will be prompted for some input which is then passed on to the playwright.ps1 Powershell script.

docker-test-vscode.mp4

Example running the tests in UI mode using the Visual Studio Code task

When you run the run tests ui task you will be prompted for some input which is then passed on to the playwright.ps1 Powershell script.

docker-test-ui-vscode.mp4

Example running the app and then the tests using the Visual Studio tasks

If you have the target test app running before you run the run tests task or run tests ui task and choose auto or from-host to the Which Playwright Web Server to use? prompt, notice that the docker container won't have to build and run the app. Instead, it immediately starts to run the tests against the the target test app that is running on the host.

docker-test-with-host-webserver-vscode.mp4

Example debugging the app using Visual Studio Code

Although this is not related with running Playwright in docker, this demo also shows how you can configure a launch task for Visual Studio Code that let's you debug the app in Visual Studio Code.

docker-debug-vscode.mp4

Footnotes

  1. feat(webpackDevServer): Add watchOptions for webpackDevServer #1814 ↩

  2. Nodemon and webpack-dev-server hot reload not working under WSL 2 after Windows 10 resinstall ↩