Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress screenshot and video size are restricted to 800 x 600 in Chrome Headless #6210

Closed
zztessema opened this issue Jan 21, 2020 · 11 comments · Fixed by #6440 · May be fixed by rangle/test-automation#29
Closed

Cypress screenshot and video size are restricted to 800 x 600 in Chrome Headless #6210

zztessema opened this issue Jan 21, 2020 · 11 comments · Fixed by #6440 · May be fixed by rangle/test-automation#29
Assignees
Labels

Comments

@zztessema
Copy link

zztessema commented Jan 21, 2020

Current behavior:

When running cy.screenshot(), the screenshot differs when running in Chrome Headless vs Chrome Headed vs Electron Headless:

Electron Headless:
test1

Chrome Headed:
test1

Chrome Headless:
test1

Desired behavior:

It seems as though any screenshot taken while running headless chrome will produce an image that is only 800x600 pixels. This is an issue because i would like to run all tests using headless chrome, however the screenshots headless chrome produces are not large (wide/tall) enough to fully capture my application.

When running chrome headless, changing the size using cy.viewport only works when the given size is less than 800x600. For example if i call cy.viewport(1920,1080) then call cy.screenshot(), the screenshot will be 800x600. However if i call cy.viewport(700,500) then call cy.screenshot(), the screenshot will be 700x500.

Test code to reproduce

cy.visit('https://google.com/')

cy.screenshot('test1') //produces a 800x600 pixel image

cy.viewport(700,500)
cy.screenshot('test2') //produces a 700x500 pixel image

cy.viewport(1920,1080)
cy.screenshot('test3') //produces a 800x600 pixel image

Versions

Cypress 3.8.2
Windows 10
Resolution 1920x1200
node 12.13.1
npm 6.12.1

@jennifer-shehane
Copy link
Member

I do see this sizing discrepancy noted in the original PR fixing the chrome headless here: #5953 (comment), but this is not documented anywhere in our documentation and I would definitely find this confusing as a user.

@flotwig can you comment on why there is this limitation, if there is a workaround? I'd like this documented if so.

@jennifer-shehane jennifer-shehane changed the title Cypress screenshots differ when running in Chrome Headless Cypress screenshot and video size are restricted to 800 x 600 in Chrome Headless Jan 28, 2020
@flotwig flotwig self-assigned this Jan 28, 2020
@flotwig
Copy link
Contributor

flotwig commented Jan 28, 2020

I think there was a gap in the tests for screenshots taken via Chrome headless, because I have not noticed this before.

I wonder if #5899 would fix this.

@Threnos
Copy link

Threnos commented Jan 29, 2020

Faced this issue as well.
Debugging failing headless tests became harder - such small wiewport prevents from reading error messages.

@steveharrison
Copy link

steveharrison commented Jan 30, 2020

I fixed this issue by overriding the Chrome window size with the largest viewport used in my tests (in my case, the Cypress default viewport of 1000x660px) in the before:browser:launch hook.

plugins/index.js

module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
      args.push('--window-size=1000,660');
      return args;
    }
});

I can also just run Chrome without --headless inside Docker, because then the Chrome window size will be 1050x1004px.

I am a little bit confused about how the viewport works in Cypress though: I've read stuff about the XVFB viewport, Chrome viewport, and differences between headed/headless. From a user perspective, shouldn't Cypress make sure the XVFB/Browser viewport always matches what has been set in cy.viewport? If the XVFB/Browser viewport can only be set when starting up, could Cypress scan for the largest cy.viewport value and automatically use that value?

@Threnos
Copy link

Threnos commented Jan 30, 2020

@steveharrison I'd say, that there's no need for such complex actions like scanning and guessing which resolution is larger etc. The most simple and sturdy solution would be to provide configurations for browser window size, e.g. browserWidth/Height.

@cooleiwhistles
Copy link

cooleiwhistles commented Jan 30, 2020

I fixed this issue by overriding the Chrome window size with the largest viewport used in my tests (in my case, the Cypress default viewport of 1000x660px) in the before:browser:launch hook.

plugins/index.js

module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
      args.push('--window-size=1000,660');
      return args;
    }
});

I can also just run Chrome without --headless inside Docker, because then the Chrome window size will be 1050x1004px.

I am a little bit confused about how the viewport works in Cypress though: I've read stuff about the XVFB viewport, Chrome viewport, and differences between headed/headless. From a user perspective, shouldn't Cypress make sure the XVFB/Browser viewport always matches what has been set in cy.viewport? If the XVFB/Browser viewport can only be set when starting up, could Cypress scan for the largest cy.viewport value and automatically use that value?

This worked to have screenshots in expected size, but often times I see this error now (with or without the workaround)
cy.screenshot() timed out waiting '30000ms' to complete.

@0xIslamTaha
Copy link

0xIslamTaha commented Feb 3, 2020

I have this as a configuration in the plugin/index.js

  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
        args.push('--window-size=1280,1024');
  
        // whatever you return here becomes the new args
        return args
    }

But I can see that all tests gonna fail if I run it with --headless mode and here is the terminal record

asciicast

and here is the error messages in headless mode

Error: Image size (1272x720) different than saved snapshot size (1280x720).
See diff for details: /opt/code/github/cloud-fe-components/tests/visual_testing/snapshots/stepper.js/__diff_output__/test001_stepper_at_step_1.diff.png
    at Context.<anonymous> (http://localhost:6006/__cypress/tests?p=tests/visual_testing/support/index.js-495:52:17)

even trying to set the xvfb values doesn't help

xvfb-run --server-args="-screen 0, 1280x1024x24" ./node_modules/.bin/cypress run --spec tests/visual_testing/integration/stepper.js -b chrome --headless

Update

  • I run the test cases inside the docker container from cypress/browsers:node12.13.0-chrome78-ff70 images, and everything going well. here is my execution command
    npx cypress run -b chrome --headless

I'm not sure why running the --headless mode in the host machine (ubuntu 19 os), raises an error

@cypress-bot cypress-bot bot added the stage: needs investigating Someone from Cypress needs to look at this label Feb 4, 2020
@jennifer-shehane jennifer-shehane added the type: unexpected behavior User expected result, but got another label Feb 4, 2020
@jennifer-shehane
Copy link
Member

jennifer-shehane commented Feb 7, 2020

So, after talking this over with the team. This is kind of expected behavior. Basically, Chrome is running headless - which means there is no display. So, there is not a display size that Cypress can pull from and automagically know to set the window size at.

However, I advocated for having a better default than 800 x 600, which we can do. Since xvfb generally restricts the size of most users runs during cypress run to 1280 x 720, I thought this would be a better default.


@cooleiwhistles The cy.viewport() only applies to the height and width of the application under test when rendered. Not the size of the display (including the command log and Cypress Test Runner GUI).

@0xIslamTaha @cooleiwhistles I am not able to recreate any timeouts during cy.screenshot() while passing the window size using the basic Cypress screenshot command (no plugins). Are you using a plugin to take screenshots? Because this may be an issue with the plugin or it could be related to this issue #5016

Workaround

You can set the window size to whatever you want in Chrome headless.

3.x.x versions

module.exports = (on, config) => {
  on('before:browser:launch', (browser, args) => {
    if (browser.name === 'chrome' && browser.isHeadless) {
      args.push('--window-size=1280,720')
  
      return args
    }
  })
}

4.x.x version

module.exports = (on, config) => {
  on('before:browser:launch', (browser, launchOptions) => {
    if (browser.name === 'chrome' && browser.isHeadless) {
      console.log('TRUE')
      launchOptions.args.push('--window-size=1280,720')
  
      return launchOptions
    }
  })
}

Code

Default args we send to Chrome are here: https://github.com/cypress-io/cypress/blob/develop/packages/server/lib/browsers/chrome.ts#L42:L42

So, I guess you would have to somehow detect that they are running Chrome headless and push in the --window-size=1280,720 arg when that is true.

I think you would likely do this check in this method: https://github.com/cypress-io/cypress/blob/develop/packages/server/lib/browsers/chrome.ts#L341:L341

@cypress-bot cypress-bot bot added stage: ready for work The issue is reproducible and in scope and removed stage: needs investigating Someone from Cypress needs to look at this labels Feb 7, 2020
@Threnos
Copy link

Threnos commented Feb 7, 2020

@jennifer-shehane Thank you so much for the workaround.
I will try it as soon as possible.

@cypress-bot cypress-bot bot added stage: needs review The PR code is done & tested, needs review and removed stage: ready for work The issue is reproducible and in scope labels Feb 14, 2020
@brian-mann brian-mann self-assigned this Feb 14, 2020
@cypress-bot cypress-bot bot added stage: work in progress and removed stage: needs review The PR code is done & tested, needs review labels Feb 27, 2020
@cypress-bot
Copy link
Contributor

cypress-bot bot commented Feb 28, 2020

The code for this is done in cypress-io/cypress#6440, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@cypress-bot
Copy link
Contributor

cypress-bot bot commented Feb 28, 2020

Released in 4.1.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v4.1.0, please open a new issue.

@cypress-bot cypress-bot bot locked as resolved and limited conversation to collaborators Feb 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
8 participants