-
-
Notifications
You must be signed in to change notification settings - Fork 6.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Targeting more than JSDOM #848
Comments
Thanks for opening this issue. I'll triage some older issues that are related to this task and close them in favor of this fresh discussion. Browser testing is part of my plan for Jest, however it isn't something we are planning to use at FB so it doesn't have a high priority. This is something where I'd be excited to support someone from the community – this is certainly a challenging task but it would make Jest even more compelling to people :)
This would be easy if it wasn't for the fact that this callback would be async but Jest relies on synchronous calls to At that point, here is what we'll need to do:
|
As a community member, is there a component here that would be best for me to work towards? I'm still familiarizing myself with Jest internals, but it seems like the bulk of the work has to be done in One of the attractive parts of Karma is how easy it is to integrate browserify and webpack. It is not difficult to test code bundled in the same way between test and dev. This is attractive to me. While not a direct concern to this issue, I am curious if Jest could integrate with these tools. This is a difficult problem. Looking forward to digging in. |
I think we'll first need to think pretty hard on how to implement this in a way that it's gonna work well before we start writing any code. Once we have a good idea of what we are gonna build here, I can think more about the staging and smaller chunks of work. I think one of the big questions is how to create the bundle. webpack/browserify will be insufficient for a few reasons:
I'm all for integrating with existing tools, but it seems that we need the ability to let Jest do it's runtime work of sandboxing and shipping its own require implementation. Another option would be to make the runtime parts of Jest work in the browser and consider them as entirely separate from the cli runner. One example is the modularization we are doing already: |
been thinking about this a bit. Mocking is one of the big features of Jest over other options., this in particular makes a browser runner hard since the execution environment really needs to understand modules. to that end I think using webpack, under the hood, makes a lot of sense. webpack can more than handle the resolution and mocking concerns, with the right set of plugins (which can consume jest-resolve). the suggestion tho, is to use it as an implementation detail/node environment polyfill. i need to spend more time consolidating my thoughts (just typing this up quickly on my phone) but I think the big bits that would be needed in jest is pluggable environments (jsdom, browser) and an environment agnostic runtime/runner. I suppose similar to karma s launchers but maybe less generic |
@jquense I don't want to run the risk of repeating you too much but I wanted to share some thoughts too.
I wonder what the minimum required surface area would be for browser execution. What happens when Could mocking be handled within node, leaning on existing work, with most code execution happening through passing a bundle of javascript to a browser? Could this be as simple as preparing a bundled javascript file, injecting that into a browser context, and reporting back results? I think this is how Karma's test harness works. I gravitate towards keeping this as simple as dumping JS on the page and tracking console.log/window.onerror. Though I'm sure it's more complicated than that. |
karma requires a preprocessor for any sort of modular code, such as karma-webpack.
this is what im suggesting, but the above would require building a file concatenator based on module/depedency graphs with custom resolution logic. essentially webpack :P i was thinking we could save a ton of hard work by leaveraging webpack instead of builing something. the rest is figuring out api boundries between the test runner, environment and test providers |
I think that there is a big difference between the feature set of something like Jest vs karma. karma is a simply a pluggable runner/framework for running a set of tests in many different environments. Jest on the other hand makes a lot of assumptions about the environment. Jest has wider scope than karma; its more like karma + jasmine/mocha + jsdom-launcher + karma-webpack + rewire/inject-loader. Because of this I don't think that a super flexible launcher api like Karma's is well suited to Jest. To that end I think that what you really need for jest-browser to work is the ability to create a node like environment in the browser, to which I say that webpack is the most robust and flexible tool for doing this already out there. It's plugin system can more than handle the concerns made by @cpojer above, as well as all the jest config options. Undoubtable it'd be slower, but I think that could be isolated to the browser-runner bits so that fb wouldn't mind ;P As soon as a get a bit of time i want to see where the sore points are in the existing jest infra, and where assumptions are being made about the runtime. I think that will put us in a good place to talk about how the API's would need to change in order to support this sort of use case. I feel pretty optimistic thou, the team as already done an amazing job as modularizing and extracting jest into lots of its constituent bits :) |
I appreciate this discussion got started again. To me this has always been exciting but also risky for the project. I'll support this as much as I can and have some thoughts as well. My ideal solution to this would be somewhere in jest-runtime and jest-environment-*. The problem is that right now; Jest requires and compiles files "just-in-time" when they are required which means they need to be synchronous. Because of the mocking feature, the entire module system and file system needs to be virtualized in the browser, as well as the entire transform pipeline with babel. However, there are two more ways of solving this, both equally exciting:
This would run on the client; when we call into require we'd request the transformed module over a websocket connection. |
@cpojer I'm actually not into the topic but in case you missed it, async |
@thymikee yeah good point but that also requires to write a transform from require to async import, it's very similar I guess. |
i'm gonna guess that requesting files over the wire (even locally) is going to be too slow. My thought was that just shipping all the mocks for required packages would be simpler and probably faster overall? Maybe i'm missing something here but most jest mocks are statically knowable, right? detecting calls to
This speaks to my earlier point, its a tough one. Writing a module env for the browser is going to be super fraught and complicated. The thought was to leavage work already done on that front via webpack, for this bit. Especially since almost every nook and crany of webpack is pluggable. |
Sorry if you guys have thought of this already but if the
and let webpack just include them both? edit |
@thattomperson jest-runtime does this branching already. |
Playing around with this a bit more. I've been experimenting with running Jest inside of an Electron browser instance: https://gist.github.com/nhunzaker/5fc6f3ea153319e3140275914773d5bc Electron is pretty neat because I can just require Jest inside of the browser. Trouble is that I don't have encapsulation. Test globals leak into each other. Ideally, I think it would be neat if each test suite opened an isolated browser instance. I'm not sure if there's really any benefit here. Eventually I think it would be neat if you could mount a React app to the browser for testing, stepping through breakpoints and watching behavior play out in a real browser. No real questions here, but maybe it gives someone else some inspiration. |
Wow, this is super cool you're working on this! I think you should spawn new browser in |
oh hell yeah @nhunzaker. I'm extremely excited about this. Do you have a guide to set this up? If @thymikee's suggestion doesn't work, could you use iframes in |
@cpojer I've put up the code here: https://github.com/nhunzaker/jest-electron-environment I think I could use the What I'm struggling with is how to get the actual code for the test: ☝️ I'm not too familiar with Jasmine script contexts (if I have that right). Is there a way to get the string content for the test? |
While running Jest in an electron shell is an interesting project, I think it only solves part of the issue for real world developers. While a CLI runner or an electron environment is great for our developer workflow, we have also found many issues over the years by running our unit tests in real browsers (especially older browsers) that we didn't find in those CLI environments or Chrome alone. I think we still need a way to run Jest in real browsers, even if the electron environment is successful (and I hope it is!). |
I completely agree. As someone less familiar with Jest internals, I found Electron to be an attractive target. Electron just works, and opens up a lot of possibilities, such as recording test sessions and stepping through browser behavior using a debugger. Still, I don't want to express neglect for real browsers with that enthusiasm. Unfortunately, I also haven't been able to make the time for those problems. It would be cool to establish a more concrete roadmap for how we can get to testing in real browsers. I still have a lot to learn. |
My two cents: I got Jest running in NW.js, which is pretty similar to Electron, but it doesn't have the same node/browser context separation, so it's pretty easy to get the test code. Proof of concept here: https://github.com/suchipi/jest-node-nw-example It's using the same browser globals every run right now, but creating new iframes for each test shouldn't be too hard |
ChromeHeadless is now available, with support on it's way to karma-chrome-launcher: karma-runner/karma-chrome-launcher#111 |
I've been cooking up something out in the woods in Washington, and it's this library: https://github.com/joelgriffith/navalia. This tool has the capacity to write tests in a high-level browser API, and can parallelize them in a single browser context (think of tabs). Once I harden the API a bit I'm going to write some content on how to achieve this wizardry, and folks can hopefully begin to have a better experience with E2E-style tests. Here's a short post on how it can instrument chrome for coverage (for those of you who are curios about the API): https://codeburst.io/capturing-unused-application-code-2b7594a9fe06 |
One thing I haven't seen mentioned in this discussion (probably missed it from previous issues) is that one of one of the big wins I've seen before with browser based tests is the ability to run them on various platforms (IE/Edge, iOS on Safari, etc) to validate edge cases in those environments. It certainly doesn't cover everything you can do with automation tools, but it's been helpful to me in the past. Getting it working on Electron or NW.js would be great from a debugging perspective, but actually being able to run it on the target platform would be even better |
https://github.com/facebook-atom/jest-electron-runner worked really well for me. |
Running Jest test code on browsers using Karma is just as easy as configuring Karma to use with Jasmine as usual and adding the following lines above where the tests are loaded. window.jest = require('jest-mock');
window.expect = require('expect');
// if you are using snapshot testing, which cannot be run on browsers
expect.extend({
toMatchSnapshot: () => ({ pass: true }),
// ...
}); See https://github.com/kimamula/jest-karma-angular-demo for demo. What I want to do is to run tests usually on JSDOM (as it is faster) but sometimes on browsers to make sure my code is compatible with target browsers, which is achieved by the above setup. |
@suchipi Thanks for your feedback. |
There is a karma-jest plugin. However, it doesn't have a stable release yet. |
This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days. |
comment |
Shout out to https://github.com/kayahr/jest-electron-runner to continue the archived |
This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days. |
Activity |
This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days. |
bump |
Regarding facebook/react#5703, I wanted to open up conversation about the possibility of supporting JavaScript environments beyond JSDOM.
What does this look like?
Does it involve leaning on tools such as Karma, browser-run, or zuul?
Could it be as simple as providing a way to package a fully-encapsulated test as an environment-agnostic payload?
In any case. I thought it was worth starting discussion here.
The text was updated successfully, but these errors were encountered: