feat: add cypress end to end tests #725

CharlesSheelam · 2024-05-27T15:11:58Z

Working on smoke testing to test functionality of the comses website.

should call e2e tests something else, functional? smoke? browser? integration? add initial smoke tests

alee

Comments attached, what do you think @sgfost ?

Makefile

e2e/cypress/fixtures/codebase/testNarrativeDocumentation.txt

e2e/cypress/fixtures/codebase/testSourceCode.txt

alee · 2024-07-19T23:14:07Z

e2e/cypress/tests/beforeLoginTests.spec.ts

+  it("should browse the model library and be able to enter a search query", () => {
+    cy.visit("/"); // visit homepage
+    cy.visit("/codebases");
+    assert(cy.get("h1").contains("Computational Model Library"));


should we also make assertions on an expected model to find in the model library?

alee · 2024-07-19T23:14:59Z

e2e/cypress/tests/beforeLoginTests.spec.ts

+    // Navigate to the first 'search-result', find the 'a' tag within the 'title' div, and click it
+    cy.get(".search-result", { timeout: 10000 })
+      .first()
+      .find(".title a")


this filtering criteria seems poorly vague for identifying a codebase download. Setup a more precise criteria even if you need to include additional metadata on the model library page.

alee · 2024-07-19T23:15:42Z

e2e/cypress/tests/codebase.spec.ts

+
+  it("should  be able to download a codebase", () => {
+    cy.visit("/codebases");
+    cy.get(".search-result").first().find("a").first().click();


How brittle is this criteria? Is the first a href after a .search-result CSS class targeted enough to always get the first codebase? Or should we make all codebase results more explicit with a CSS class or identifier and target that explicitly?

I think this and the above comment are the same issue, should probably just use data attributes to target this

alee · 2024-07-19T23:18:58Z

e2e/cypress/tests/codebase.spec.ts

+  it("should verify that the codebase was uploaded correctly", () => {
+    cy.visit("/codebases");
+    cy.get(".search-result").first().find("a").first().click();
+    assert(cy.get("h1").contains("Codebase Title"));


These kinds of assertions are highly dependent on whatever stylistic or textual changes we want to make in the frontend. It might be better to have a list or other data structure of expected text and programmatically iterate through it with assertions and comment the data structure as a place of possible future change.

Good idea, perhaps a global object that describes the initial database state which can be modified at runtime? for example:

{ ... codebases: [ { title: "Codebase Title", description: "something" }, ], users: [ { username: "test_user", password: "123456", ... } ], ... }

and then, as Allen said, use this to assert that certain things are found on a page, rather than at some specific location in the DOM tree

sgfost

thanks @CharlesSheelam. I think the main thing remaining is fragility in selectors.

For cases where we need to select an element to perform some action (like grabbing a codebase search result and the link to it's detail page) it should look for a data attribute rather than a css class/first element/etc.

And for cases where we're asserting that something exists, I would say just expand the search to be much more broad (whole page or perhaps an existing container with a data attribute) and use a global data structure that describes the content of the database to define what we want to check for

e2e/cypress/tests/user.spec.ts

e2e/cypress/tests/codebase.spec.ts

runs in a test environment with built frontend assets and a completely separate db container (is there a more elegant way to achieve isolation here? more similar to django's test db setup?) always restores from minimal borg repo actual cypress tests are run with cypress github action and on failure, screen recordings are saved as artifacts

sgfost · 2024-07-24T02:00:03Z

tests are failing when running on the GH runner, partially because I disabled the debug toolbar but there are a few other things it seems to be getting tripped up on. They mostly seem to coincide with selectors mentioned above that should get changed anyways

…ource code

* setup shouldn't be run as a test * fixed a couple fragile selectors by removing dependencies on classes/ids/element trees subject to change

sgfost · 2024-08-05T19:42:32Z

@CharlesSheelam I did a little bit of cleanup/refactoring. Just a couple things before I think this is good to go:

the data fixture should include things that already exist in the db. The things that are going to get added can stay in, perhaps its a bit cleaner that way, but they'll need to be separated somehow from the data structure that represents the starting data
still some fragile selectors that use things that are subject to change like css classes. I redid a couple of these (license selection in codebase.spec.ts, name inputs in user.spec.ts) but there a few more instances scattered around that use stuff like bootstrap button/card classes

- apply prettier - fix industry options selector - apply style to frontend

alee

LGTM, only issue is the data-cy attribute values are sometimes space cased or kebab-cased, should stick to one convention. Can fix in a future refactor

alee · 2024-08-29T05:57:11Z

e2e/cypress/support/util.ts

+ * Get an element by its data-cy attribute
+ * @param {string} value - value of the data-cy attribute
+ */
+export function getDataCy(value: string): Cypress.Chainable {


I get why this is named because it retrieves the "data-cy" attribute but getCypressData(attributeValue: string) might be a better name overall. To be considered by future devs, not a blocker on this PR

alee · 2024-08-29T05:57:55Z

e2e/cypress/tests/codebase.spec.ts

+      getDataCy("codebase title").type(codebase.title);
+      getDataCy("codebase description").type(codebase.description);
+      getDataCy("codebase replication-text").type(codebase["replication-text"]);
+      getDataCy("codebase associated publications").type(codebase["associated-publications"]);


inconsistency here, why is replication-text hyphenated if "associated publications" is not? based on the rest of the attribute names I think we should either use spaces or kebab-case for all of em

sgfost · 2024-09-03T17:29:34Z

failing e2e tests is due to programming error on my part in #757. I suppose it illustrates the usefulness of these!

alee

thanks @sgfost and @CharlesSheelam !

* removed unused class attr

CharlesSheelam force-pushed the smoke-testing branch 2 times, most recently from 30d651d to d0dec6b Compare June 24, 2024 22:24

sgfost and others added 5 commits July 16, 2024 13:45

test: refresh minimal e2e test setup

85c6b0d

should call e2e tests something else, functional? smoke? browser? integration? add initial smoke tests

feat: add smoke tests for jobs page

ba366bf

feat: add smoketests for downloading / upload codebases

0c23782

fix: add data-cy attributes to each element for more accurate selectors

2e8049c

feat: add final smoke tests

273ee20

CharlesSheelam force-pushed the smoke-testing branch from cedfe45 to 273ee20 Compare July 16, 2024 20:48

alee changed the title ~~Feat: Smoke testing~~ feat: add cypress end to end tests Jul 17, 2024

alee requested changes Jul 19, 2024

View reviewed changes

sgfost requested changes Jul 22, 2024

View reviewed changes

e2e/cypress/tests/user.spec.ts Outdated Show resolved Hide resolved

e2e/cypress/tests/codebase.spec.ts Outdated Show resolved Hide resolved

CharlesSheelam and others added 5 commits August 3, 2024 12:42

fix: modify tests to pass

5f09633

fix: use more accurate selectors for codebase & upload zip file for s…

b133e27

…ource code

feat: add testCodebase.zip

5baa648

fix: pull data from json object instead of hardcoding

64c0298

chore: remove unused cypress configs and migrations

c668739

sgfost force-pushed the smoke-testing branch from 5bf1330 to 2c1bba2 Compare August 5, 2024 19:26

refactor: use helper for data-cy selectors

1fa7793

* setup shouldn't be run as a test * fixed a couple fragile selectors by removing dependencies on classes/ids/element trees subject to change

sgfost force-pushed the smoke-testing branch from 2c1bba2 to 1fa7793 Compare August 5, 2024 19:40

CharlesSheelam force-pushed the smoke-testing branch from 0334f94 to 2fe98f6 Compare August 19, 2024 17:42

refactor: use data-cy selectors instead of css attributes

11bcf8e

- apply prettier - fix industry options selector - apply style to frontend

CharlesSheelam force-pushed the smoke-testing branch from 589389b to 11bcf8e Compare August 28, 2024 21:19

alee approved these changes Aug 29, 2024

View reviewed changes

alee force-pushed the smoke-testing branch from 685b310 to d964e32 Compare September 1, 2024 23:45

sgfost approved these changes Sep 3, 2024

View reviewed changes

alee approved these changes Sep 3, 2024

View reviewed changes

CharlesSheelam and others added 2 commits September 3, 2024 16:20

refactor: only use kebab-case + add comments

e1c488c

test(e2e): server should wait for vite to build

ade645d

* removed unused class attr

alee force-pushed the smoke-testing branch from b8d086c to ade645d Compare September 3, 2024 23:20

alee merged commit 99c7478 into comses:main Sep 3, 2024
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add cypress end to end tests #725

feat: add cypress end to end tests #725

CharlesSheelam commented May 27, 2024

alee left a comment

alee Jul 19, 2024

alee Jul 19, 2024

alee Jul 19, 2024

sgfost Jul 22, 2024

alee Jul 19, 2024

sgfost Jul 22, 2024

sgfost left a comment

sgfost commented Jul 24, 2024

sgfost commented Aug 5, 2024

alee left a comment

alee Aug 29, 2024

alee Aug 29, 2024

sgfost commented Sep 3, 2024

alee left a comment

feat: add cypress end to end tests #725

feat: add cypress end to end tests #725

Conversation

CharlesSheelam commented May 27, 2024

alee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sgfost left a comment

Choose a reason for hiding this comment

sgfost commented Jul 24, 2024

sgfost commented Aug 5, 2024

alee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sgfost commented Sep 3, 2024

alee left a comment

Choose a reason for hiding this comment