Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add end to end tests + benchmark across browsers #7

Closed
Divide-By-0 opened this issue Dec 20, 2022 · 1 comment
Closed

Add end to end tests + benchmark across browsers #7

Divide-By-0 opened this issue Dec 20, 2022 · 1 comment
Assignees
Labels

Comments

@Divide-By-0
Copy link
Member

Start with a simple Puppeteer end-to-end test that pastes a valid email and address in, generates a proof, and verifies the proof. Benchmark this test on Browserstack to see where this code does and doesn't work.

@Divide-By-0 Divide-By-0 changed the title Add end to end tests Add end to end tests + benchmark across browsers Dec 20, 2022
@Divide-By-0
Copy link
Member Author

Divide-By-0 commented Mar 8, 2023

rmccrear, Re: "many browsers" testing i.e. https://www.lambdatest.com/online-browser-testing ?

From what I understand about lambdatest, is that it requires a deployed site,
then you give lambdatest your deployed url and the tests, and it runs the tests against the deployment.

You could set up render.com to deploy on each commit to main. Then have lambdatest run against your zkemaill.onrender.com (deployed as a static website).

So the workflow could look like this:

  1. pre-commit: npm run tests:e2e --> get your local results for firefox, chromium webkit printed to command line (and to the junit.xml files)
  2. on PR...run the tests in three browsers on circleci to record results for firefox, chromium, webkit (but you might need a paid account to run cpu intensive tests)
  3. On merge to main, automatically deploy to render.com, then run lambdatest or browserstack to run on the environments you choose. (Probably need a paid account.)

But, if firefox, chromium, webkit are enough for you, then you won't need lambdatest/browswerstack, and could get away with just local/circleci. In that case you would just do 1) and 2) and skip 3).

There are a few problems with running perf tests on the cloud, though. One is that you won't have complete control over the instances. You won't know what chip they are running, you are sharing the machine which the VM runs with others, the provider might allow you to "burst" which would change the results. So, pass/fail tests would work fine, but I'm not so sure perf testing on these platforms would be accurate. (I can't find any services that specifically advertise perf testing for web apps.)


I tried to setup the tests on LambdaTest. They ran up until the zkp started running, then get cut off with signal=SIGSEGV.
I actually had a call with a demo sales rep, and he said that something long running like that probably wouldn't run on their servers. You can contact sales about that, if you want to pursue it further. Sorry about that! But, here is a PR with the LambdaTest, if you want to try it yourself.

The PR is here: #37

You just set the env in your bash profile before running yarn test:

export LT_USERNAME="username"
export LT_ACCESS_KEY="lksajdflkjadsfyourkey"

cd e2e-lambdatest
yarn
yarn test

Then go to the LambdaTest console to take a look at the screen recording.

Done with #36

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants