Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI: use Jest to test CLI commands and remove outdated fixtures #12936

Merged
merged 19 commits into from
Nov 30, 2020
Merged

Conversation

jamesgeorge007
Copy link
Member

@jamesgeorge007 jamesgeorge007 commented Oct 28, 2020

What I did

  • Removed CLI test fixtures.
  • Leverage Jest for e2e tests.

closes #13318

@jamesgeorge007 jamesgeorge007 force-pushed the e2e-tests branch 3 times, most recently from 7a7ff2f to 45f9f2f Compare October 29, 2020 07:25
@jamesgeorge007
Copy link
Member Author

Running smoke-test in sfc_vue directory exhibits an inconsistent behavior.

Starting storybook manually works fine (process completes with exit code 0):-
Screenshot 2020-10-29 at 1 37 50 PM

It fails when invoked via jest:-
Screenshot 2020-10-29 at 1 32 57 PM

/cc @shilman @ndelangen

@jamesgeorge007 jamesgeorge007 marked this pull request as ready for review October 29, 2020 18:18
@shilman shilman added the maintenance User-facing maintenance tasks label Nov 1, 2020
lib/cli/package.json Outdated Show resolved Hide resolved
lib/cli/test/commands/init.test.js Outdated Show resolved Hide resolved
lib/cli/test/helpers.js Outdated Show resolved Hide resolved
@gaetanmaisse
Copy link
Member

Thanks for the PR @jamesgeorge007 💪🏻

I just pushed a small commit to fix a path, broken after a directory move, and left a few comments.
Can you take a look at them? Let me know if something is unclear :)

I will take a look at the sfc_vue case 🤔

@jamesgeorge007 jamesgeorge007 force-pushed the e2e-tests branch 2 times, most recently from 1026ab9 to b85a883 Compare November 1, 2020 18:30
@gaetanmaisse
Copy link
Member

gaetanmaisse commented Nov 2, 2020

@jamesgeorge007 I'm focusing on another PR for now but will take a new look at this one soon ;)

@gaetanmaisse
Copy link
Member

gaetanmaisse commented Nov 18, 2020

Found the issue with some fixtures! I will add commits to fix that in the next few days :)

@gaetanmaisse
Copy link
Member

#13231 should fix things but a new error appeared, I'm investigating a bit more 🔍

@jamesgeorge007 jamesgeorge007 force-pushed the e2e-tests branch 3 times, most recently from b6611c7 to 922e9c3 Compare November 26, 2020 08:03
@jamesgeorge007 jamesgeorge007 marked this pull request as draft November 26, 2020 13:44
@gaetanmaisse
Copy link
Member

CLI tests are back to 🟢 on next so it should be here too. Can you update the branch?

To get the full result of the tests and not stop at the first failing one is really great 🎉 !

I think one more thing we need to improve is maybe how the underlying error/logs can be catch and display clearly in Jest test result. To have something more meaningful than:

    expect(received).toBe(expected) // Object.is equality

    Expected: 0
    Received: 1

@jamesgeorge007
Copy link
Member Author

jamesgeorge007 commented Nov 27, 2020

Running smoke-test in the sfc_vue directory is still failing.

Screenshot 2020-11-27 at 9 56 40 PM

@gaetanmaisse
Copy link
Member

I discussed with @ndelangen and @shilman and it looks like all the fixtures are outdated. And so the result of these tests aren't really relevant nor reflect if Storybook properly works or not. As keeping up to date the fixtures is very time-consuming and as we have more robust e2e tests, it looks like we will remove these ones: #13318.

I'm a bit disappointed to tell you that although you have spent time to them 😕

@jamesgeorge007 jamesgeorge007 marked this pull request as ready for review November 28, 2020 09:13
Copy link
Member

@gaetanmaisse gaetanmaisse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@gaetanmaisse gaetanmaisse changed the title tests(cli): migrate to jest for e2e tests CLI: use Jest to test CLI commands and remove outdated fixtures Nov 28, 2020
@gaetanmaisse
Copy link
Member

@shilman can I merge this one or are we still merging only PRs stabilizing 6.1?

@ndelangen
Copy link
Member

should be good to merge to next IMHO

@ndelangen ndelangen merged commit 0e56733 into next Nov 30, 2020
@ndelangen ndelangen deleted the e2e-tests branch November 30, 2020 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cli maintenance User-facing maintenance tasks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants