Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/2141 cypress #2305

Merged
merged 35 commits into from
Jan 27, 2023
Merged

Chore/2141 cypress #2305

merged 35 commits into from
Jan 27, 2023

Conversation

jtimpe
Copy link

@jtimpe jtimpe commented Dec 7, 2022

Summary of Changes

Provide a brief summary of changes
Pull request closes #2141

How to Test

List the steps to test the PR
These steps are generic, please adjust as necessary.

# in separate terminals
cd tdrs-frontend && npm run start
cd tdrs-backend && docker-compose up

# in another terminal
docker-compose exec web python manage.py generate_cypress_users
cd ../tdrs-frontend
npm run test:e2e
  1. Run the account.feature test file

Deliverables

More details on how deliverables herein are assessed included here.

Deliverable 1: Accepted Features

Checklist of ACs:

  • Determine if we're interested in smoke testing or more robust integration testing
  • Define some criteria for what makes a "good" test given the end goal
  • Research available test runners and/or open source or SaaS tools for each type, select a tool to use
  • Start an automated suite by setting up the test runner(s)
  • Set up a test suite using the selected runner. Create automated test(s) for an authenticated "hello world"
  • Document the research criteria and decision in an ADR
  • Update documentation with instructions for running tests locally and in each environment
  • lfrohlich and/or adpennington confirmed that ACs are met.

A couple of relatively easy nice-to-haves

Refer to Cypress's CircleCI integration guide

Deliverable 2: Tested Code

  • Are all areas of code introduced in this PR meaningfully tested?
    • If this PR introduces backend code changes, are they meaningfully tested?
    • If this PR introduces frontend code changes, are they meaningfully tested?
  • Are code coverage minimums met?
    • Frontend coverage: [insert coverage %] (see CodeCov Report comment in PR)
    • Backend coverage: [insert coverage %] (see CodeCov Report comment in PR)

Deliverable 3: Properly Styled Code

  • Are backend code style checks passing on CircleCI?
  • Are frontend code style checks passing on CircleCI?
  • Are code maintainability principles being followed?

Deliverable 4: Accessible

  • Does this PR complete the epic?
  • Are links included to any other gov-approved PRs associated with epic?
  • Does PR include documentation for Raft's a11y review?
  • Did automated and manual testing with iamjolly and ttran-hub using Accessibility Insights reveal any errors introduced in this PR?

Deliverable 5: Deployed

  • Was the code successfully deployed via automated CircleCI process to development on Cloud.gov?

Deliverable 6: Documented

  • Does this PR provide background for why coding decisions were made?
  • If this PR introduces backend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces frontend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces dependencies, are their licenses documented?
  • Can reviewer explain and take ownership of these elements presented in this code review?

Deliverable 7: Secure

  • Does the OWASP Scan pass on CircleCI?
  • Do manual code review and manual testing detect any new security issues?
  • If new issues detected, is investigation and/or remediation plan documented?

Deliverable 8: User Research

Research product(s) clearly articulate(s):

  • the purpose of the research
  • methods used to conduct the research
  • who participated in the research
  • what was tested and how
  • impact of research on TDP
  • (if applicable) final design mockups produced for TDP development

@codecov
Copy link

codecov bot commented Dec 7, 2022

Codecov Report

Merging #2305 (e63e723) into develop (70f47b7) will decrease coverage by 0.49%.
The diff coverage is 21.05%.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2305      +/-   ##
===========================================
- Coverage    94.04%   93.55%   -0.49%     
===========================================
  Files           96       96              
  Lines         2853     2872      +19     
  Branches       263      265       +2     
===========================================
+ Hits          2683     2687       +4     
- Misses         123      137      +14     
- Partials        47       48       +1     
Flag Coverage Δ
dev-backend 93.34% <21.05%> (-0.67%) ⬇️
dev-frontend 94.11% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
tdrs-backend/tdpservice/users/api/login.py 88.66% <13.33%> (-6.02%) ⬇️
tdrs-backend/tdpservice/urls.py 92.59% <33.33%> (-7.41%) ⬇️
tdrs-backend/tdpservice/settings/common.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 70f47b7...e63e723. Read the comment docs.

@jtimpe jtimpe added the WIP label Dec 8, 2022
@jtimpe jtimpe added raft review This issue is ready for raft review and removed WIP labels Jan 9, 2023
Copy link
Collaborator

@andrew-jameson andrew-jameson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outside of a broken link and the email change, LGTM!

@jtimpe jtimpe requested a review from andrew-jameson January 9, 2023 20:24
Copy link
Collaborator

@andrew-jameson andrew-jameson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just need to delete that extra parenthesis and good to go.

jtimpe and others added 2 commits January 11, 2023 08:27
@andrew-jameson andrew-jameson added QASP Review and removed raft review This issue is ready for raft review labels Jan 12, 2023
@ADPennington
Copy link
Collaborator

@jtimpe can you get the owasp scan to run on this ticket?

jtimpe and others added 2 commits January 26, 2023 11:02
…integration-tests.md

Co-authored-by: Alex P.  <63075587+ADPennington@users.noreply.github.com>
…integration-tests.md

Co-authored-by: Alex P.  <63075587+ADPennington@users.noreply.github.com>

Deployments are slow and require a lot of manual testing; this can cause quite a backup during PR reviews and releases. Unit tests are valuable for fast feedback close to the code, but we are lacking broader-scale automated testing of features.

### Agile testing quadrants
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really nice intro discussion to automated testing. thank you @jtimpe


Tests should be run locally as much as possible during development to ensure changes don't unexpectedly break something else in the system. Tests should at least be run before and during the PR process, so we should add a CircleCI task to run Cypress tests along with the unit tests.

We should also regularly run tests in a deployed environment. In order to best simulate a production deployment, we should automatically deploy to a specific environment after merges to `develop`, after which the end-to-end tests should automaticfally be run. This would require a few more changes to our CircleCI configuration and processes.
Copy link
Collaborator

@ADPennington ADPennington Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏾 it'll be helpful to have a workflow diagram for this when we have this figured out. And figuring out where this will be deployed is another good topic for future discussion. To model production as close as possible, we currently have staging and prod matching at all times. id like to better understand how this testing will coincide with release branching strategy, if at all.

Copy link
Collaborator

@ADPennington ADPennington left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @jtimpe 🚀

note: i noticed new endpoints related to cypress show up in the backend owasp scans. we should keep an eye on what comes up in the nightly scans to be sure no new or noteworthy issues are raised.

@andrew-jameson andrew-jameson merged commit 60e5b05 into develop Jan 27, 2023
@andrew-jameson andrew-jameson deleted the chore/2141-cypress branch January 27, 2023 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

As technical lead, I want an authenticated-user integration test
4 participants