Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress tech memo, re: test-e2e-deployment #3196

Merged
merged 15 commits into from
Dec 10, 2024
Merged

Conversation

andrew-jameson
Copy link
Collaborator

@andrew-jameson andrew-jameson commented Sep 20, 2024

Summary of Changes

Provide a brief summary of changes
Pull request relates to #3141
Additionally, has some minor developer quality of life improvements:

  1. Task to run cypress
  2. Reduce CircleCI workflows kicked off by a commit to existing PR

How to Test

N/A

Deliverables

More details on how deliverables herein are assessed included here.

Deliverable 1: Accepted Features

Checklist of ACs:

  • N/A

Deliverable 2: Tested Code

  • Are all areas of code introduced in this PR meaningfully tested?
    • If this PR introduces backend code changes, are they meaningfully tested?
    • If this PR introduces frontend code changes, are they meaningfully tested?
  • Are code coverage minimums met?
    • Frontend coverage: [insert coverage %] (see CodeCov Report comment in PR)
    • Backend coverage: [insert coverage %] (see CodeCov Report comment in PR)

Deliverable 3: Properly Styled Code

  • Are backend code style checks passing on CircleCI?
  • Are frontend code style checks passing on CircleCI?
  • Are code maintainability principles being followed?

Deliverable 4: Accessible

  • Does this PR complete the epic?
  • Are links included to any other gov-approved PRs associated with epic?
  • Does PR include documentation for Raft's a11y review?
  • Did automated and manual testing with iamjolly and ttran-hub using Accessibility Insights reveal any errors introduced in this PR?

Deliverable 5: Deployed

  • Was the code successfully deployed via automated CircleCI process to development on Cloud.gov?

Deliverable 6: Documented

  • Does this PR provide background for why coding decisions were made?
  • If this PR introduces backend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces frontend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces dependencies, are their licenses documented?
  • Can reviewer explain and take ownership of these elements presented in this code review?

Deliverable 7: Secure

  • Does the OWASP Scan pass on CircleCI?
  • Do manual code review and manual testing detect any new security issues?
  • If new issues detected, is investigation and/or remediation plan documented?

Deliverable 8: User Research

Research product(s) clearly articulate(s):

  • the purpose of the research
  • methods used to conduct the research
  • who participated in the research
  • what was tested and how
  • impact of research on TDP
  • (if applicable) final design mockups produced for TDP development

@andrew-jameson andrew-jameson self-assigned this Sep 20, 2024
Copy link

codecov bot commented Nov 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.59%. Comparing base (8f5a452) to head (62690eb).
Report is 3 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #3196   +/-   ##
========================================
  Coverage    91.59%   91.59%           
========================================
  Files          297      297           
  Lines         8509     8509           
  Branches       621      621           
========================================
  Hits          7794     7794           
  Misses         601      601           
  Partials       114      114           
Flag Coverage Δ
dev-backend 91.42% <ø> (ø)
dev-frontend 92.82% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7247a50...62690eb. Read the comment docs.

@andrew-jameson andrew-jameson changed the title Bug 3141 minimal fix for test-e2e-deployment Cypress tech memo, re: test-e2e-deployment Nov 15, 2024
@andrew-jameson andrew-jameson added the raft review This issue is ready for raft review label Nov 15, 2024
## Method/Design

### Abstracted Gherkin Steps
Presently, many of the defined Javascript functions for a given Gherkin step are bespoke or single-use instead of abstracted and should be adapted. Additionally, it was found that sessions were lingering between Gherkin scenarios as we did not have generic `setup` and `teardown` implementations ahead of these. Sufficient utilization of abstraction within the scenarios which are now doing setup/teardown between scenarios and proper session management should result in a cleaner Cypress execution and make future additions simpler.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you provide an example of what level of abstraction your are driving towards?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or is that what the ### Abstracted utility authentication functions section is covering?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The abstracted section below is for the JavaScript side. I will revise this section to be more verbose on how I imagine we might restructure the Gherkin. Reading through it, especially without Jan's clearing of cookies, one scenario needed to be run before another, etc. Assuming we get the setup/teardown, these scenarios can be independent. I'll include psuedo-code examples as we discussed Friday I think on how tech memos ought to look.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might need to update cypress and/or cypress-cucumber-preprocessor to get these (i couldn't get them to work in the 5 minutes i spent messing with it), but we should have BeforeAll and BeforeEach hooks available

https://github.com/badeball/cypress-cucumber-preprocessor/blob/master/docs/cucumber-basics.md#hooks

@lhuxraft
Copy link
Collaborator

lhuxraft commented Dec 4, 2024

12/4: Andrew to re-request reviews

Copy link

@elipe17 elipe17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Thanks for adding the extra steps!

Copy link

@raftmsohani raftmsohani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@andrew-jameson andrew-jameson merged commit ae340de into develop Dec 10, 2024
12 checks passed
@andrew-jameson andrew-jameson deleted the bug/e2e-minimal branch December 10, 2024 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops documentation DX Developer Experience raft review This issue is ready for raft review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants