Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

type / submit in Cypress not triggering HTML validation on elements #6678

Open
brettz9 opened this issue Mar 9, 2020 · 19 comments
Open

type / submit in Cypress not triggering HTML validation on elements #6678

brettz9 opened this issue Mar 9, 2020 · 19 comments
Labels
pkg/driver This is due to an issue in the packages/driver directory prevent-stale mark an issue so it is ignored by stale[bot] topic: cy.type ⌨️ type: bug

Comments

@brettz9
Copy link

brettz9 commented Mar 9, 2020

Current behavior:

Not finding invalid elements, whether by:

  1. cy.get('<selector>:invalid')
cy.get('[data-name="name"]:invalid').should('have.length', 1);
  1. validity property
cy.get('[data-name="name"]').then(($input) => {
  return expect($input[0].validity.tooShort).to.be.true;
});

Desired behavior:

Correctly indicate that the element is indeed invalid (so that assertions can be based on this).

Test code to reproduce

These tests both fail in showing any problems with invalidity, despite the HTML element being queried having minlength attribute of 3 and our test typing only a single letter ("a"):

it('should catch invalid element by pseudo-selector', () => {
    cy.visit('http://localhost:8000/');
    const tooShortOfAName = 'a';
    cy.get('[data-name="name"]').type(tooShortOfAName);

    // Fails with or without a form click
    // cy.get('[data-name="action2"]').click();

    // Fails here
    cy.get('[data-name="name"]:invalid').should('have.length', 1);
  })

  it('should catch invalid element by JS API', function () {
    cy.visit('http://localhost:8000/');
    const tooShortOfAName = 'a';
    cy.get('[data-name="name"]').type(tooShortOfAName);

    // Fails with or without a form click
    // cy.get('[data-name="action2"]').click();

    // Fails here
    cy.get('[data-name="name"]').then(($input) => {
      return expect($input[0].validity.tooShort).to.be.true;
    });
  });

Here's the minimal repo to reproduce: (note that I've added a cypress install as I think you may want to do in your boilerplate also since your docs refer to installing cypress locally)

https://github.com/brettz9/cypress-test-tiny/tree/not-getting-invalidity

(i.e., on the not-getting-invalidity branch)

Outdated info I've also tacked on an environmental variable test (same as #2605 ?)
it('should obtain environmental variables', function () {
  cy.log(Cypress.env('secret'))
  expect(Cypress.env('secret')).to.equal('Be good');
});

...which is not consistently getting the secret despite it being set in plugins/index.js

module.exports = (on, config) => {
  config.env = config.env || {};
  config.env.secret = 'Be good';
  return config;
);

...consistent with the approach described for dynamically setting the environmental variables at https://docs.cypress.io/api/plugins/configuration-api.html#Switch-between-multiple-configuration-files or https://docs.cypress.io/api/cypress-api/env.html#From-a-plugin.

Versions

4.1.0, Mac OSX 10.15.2, Chrome 80.0.3987.132 (Official Build) (64-bit)

@brettz9
Copy link
Author

brettz9 commented Mar 9, 2020

@brettz9

This comment has been minimized.

@jennifer-shehane
Copy link
Member

So the problem here is not that it's not finding elements with a psuedo-selector of invalid, it's that the actions being performed through Cypress are not triggering the validation checks on the form for some reason.

You can use jQuery to query for input:invalid on the application under test after Cypress performs the type and even the click to submit the form and see that it finds no invalid elements.

Reproducible Example

spec.js

it('does not find invalid input', () => {
  cy.visit('index.html')
  cy.get('input').type('a')
  // cy.contains('Update').click()
  cy.get('input:invalid').should('have.length', 1)
})

index.html

<!DOCTYPE html>
<body>
    <input minlength="3">
    <button type="submit">Update</button>
</body>
</html>

Screen Shot 2020-03-10 at 2 01 45 PM

Events triggered during type.

Screen Shot 2020-03-10 at 1 51 42 PM

If you type manually into the form while the test is running, it will all of a sudden pass.

Screen Shot 2020-03-10 at 2 04 08 PM

@jennifer-shehane jennifer-shehane added pkg/driver This is due to an issue in the packages/driver directory type: bug labels Mar 10, 2020
@cypress-bot cypress-bot bot added the stage: needs investigating Someone from Cypress needs to look at this label Mar 10, 2020
@jennifer-shehane jennifer-shehane changed the title Not finding invalid elements type / submit in Cypress not triggering HTML validation on elements Mar 10, 2020
@shamgang
Copy link

shamgang commented Jun 14, 2020

In case it helps: I have 2 forms, one with a text and password input, and one with a text, password, and numerical input. In both cases only the text input exhibits this bug. The password and numerical inputs behave as expected - form validation popups are appearing. This remains true if I reorder the inputs on the page and change the order I type into them in Cypress.
Node 12.13.0
Cypress 4.8.0
Chrome 83.0.4103.97
Windows 10 Home v1903 build 18362.900

@zedrdave
Copy link

Just to add my own experience: getting this bug on a minlength validation on a password input:
when using cy.get().type() to type text that should be too short, I am able to submit the form both manually and through Cypress.

In essence, it looks as if Cypress is not sending the necessary key events for the form to perform validation.

@shin10
Copy link

shin10 commented Aug 18, 2020

I came across this issue as well. The minlength attribute will be ignored. Validation of the pattern
attribute pattern=".{6}" however will be triggered. Unfortunately the resulting message ("Please match the requested format") is very vague and rather confusing for the user. Further adding setCustomValidity manually wouldn't be my favourite option.

@sainthkh
Copy link
Contributor

sainthkh commented Sep 7, 2020

I tried form.reportValidity(). But somehow, it doesn't work. It always return true when there's an error in the form.

@yanfengliu
Copy link

According to MDN, minLength is only triggered if value is changed by user.

@brettz9
Copy link
Author

brettz9 commented Feb 2, 2021

Is this planned, e.g., on a Roadmap, or given a priority? It is a pretty fundamental need to be able to check forms, testing for validity (for those using the standard mechanisms for doing so). Thanks in advance...

@sainthkh
Copy link
Contributor

sainthkh commented Feb 2, 2021

@Brett9 Unfortunately, it's hard to find browser API that triggers form validation. Maybe, this can be solved with native events.

@brettz9
Copy link
Author

brettz9 commented Feb 4, 2021

@sainthkh : Thanks for the info and idea. If this would be best addressed by Native Browser Events, I would hope its consideration might be added to the list on #311 , @brian-mann .

@bahmutov
Copy link
Contributor

bahmutov commented Feb 4, 2021

An update: for me the validations are all working, you can see in https://glebbahmutov.com/cypress-examples/6.4.0/recipes/form-validation.html

Note: cy.submit does not check the validity of the form before submitting #14911 while clicking on the submit button does check

@cypress-bot
Copy link
Contributor

cypress-bot bot commented Feb 25, 2021

The code for this is done in cypress-io/cypress#14965, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

This change was reverted in #15302

@brettz9
Copy link
Author

brettz9 commented Feb 26, 2021

Sorry for the delay in replying...

As one example, I have:

<input type="password" autocomplete="new-password" class="form-control" required="" name="pass" id="pass-tf" data-name="reset-pass" minlength="6">

but this code fails:

      const tooShortPassword = 'a';
      cy.get('[data-name="reset-pass"]').type(tooShortPassword);

      cy.get('[data-name="reset-pass"]:invalid').should('have.length', 1); // Should be 1, but gives 0

The minimal test repo in the original post covers this and the other cases I believe are still occurring.

I.e., it's not just about the submission.

@brettz9
Copy link
Author

brettz9 commented Feb 28, 2021

#14965 seems not to actually be covering this issue. My concern is before submit, so I think this issue should still be open.

@jennifer-shehane
Copy link
Member

@brettz9 Yeah, you're right. I'm actually not able to get the original example to work with the new implementation. https://github.com/brettz9/cypress-test-tiny/tree/not-getting-invalidity?rgh-link-date=2020-03-09T04%3A34%3A03Z cc @sainthkh

  1. I don't see .submit() failing the test as the PR described it should.
  2. I don't see a way to check the conditions of the validity after submitting the form. How do you verify that the correct message is displayed for an invalid field in the example?

@sainthkh
Copy link
Contributor

sainthkh commented Mar 1, 2021

Confirmed that #14965 didn't fix the issue.

But the problem is that it cannot be fixed now. Because all validations are not created equal in browsers. As for the validations like required, mix, min, they always work whether it's a user input or not. But for those like minlength, maxlength, they are only triggered when users input the value.

If you want to test it yourself, sample code is below:

<input id="x" minlength="3" value="a" />
<input id="y" required />
cy.get('#x').then($x => expect($x[0].checkValidity()).to.be.false) // => fail
cy.get('#y').then($y => expect($y[0].checkValidity()).to.be.false) // => succeed

Native events might solve the problem. But I'm not sure about that.

@tusharAdvanced
Copy link

Is there any solution on this ? I am facing the same.

@a-bashkatov
Copy link

I could solve this problem using "realType" from cypress-real-events, then the input field is validated as with real text input. However, this solution is not suitable, for example, for Firefox.

@nagash77 nagash77 added the prevent-stale mark an issue so it is ignored by stale[bot] label Apr 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pkg/driver This is due to an issue in the packages/driver directory prevent-stale mark an issue so it is ignored by stale[bot] topic: cy.type ⌨️ type: bug
Projects
None yet
Development

No branches or pull requests