Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: mitmproxy + low bandwidth + automated QA using detox #8202

Merged
merged 186 commits into from
Oct 14, 2024

Conversation

rahimrahman
Copy link
Contributor

@rahimrahman rahimrahman commented Sep 4, 2024

Summary

Update E2E iOS Detox Test to have the ability to run in a low-bandwidth environment using similar label on PR. A new label has been introduced called E2E iOS tests for PR (LBW 1). The idea is to be able to create multiple profile (LBW 1-n) and change the bandwidth speed + ping time.

Ticket Link

https://mattermost.atlassian.net/browse/MM-60378

TODOs

  • fixing some bandwidth lowering flakiness.
  • finalize version of the action files to be reusable. Create a low-bandwidth composite/reusable workflow to be ran if the runner is set for low-bandwidth (perhaps pass using env variable?)
  • create a trigger for testing using detox on low-bandwidth. We have E2E iOS tests for PR to start the test. Maybe we can use E2E iOS tests for PR (low bw).
  • Changes to certain timeouts because the current detox test has a 1s timeout while waiting for elements to show up (or else it would consider the test to fail). With low bandwidth (at 1s ping), all test will fail, so the test timeout would have to evolve. We can just replace the timeout constant if runner is doing low bandwidth test.

Checklist

  • Added or updated unit tests (required for all new features)
  • Has UI changes
  • Includes text changes and localization file updates
  • Have tested against the 5 core themes to ensure consistency between them.
  • Have run E2E tests by adding label E2E iOS tests for PR.

Device Information

This PR was tested on:
iPhone 14, iOS 17.4 (even though was was passed is iOS 17.0)

Screenshots

Release Note


@yasserfaraazkhan yasserfaraazkhan added the E2E iOS tests for PR (LBW 1) Run iOS E2E Detox tests on Low Bandwidth label Sep 24, 2024
@github-actions github-actions bot removed the E2E iOS tests for PR (LBW 1) Run iOS E2E Detox tests on Low Bandwidth label Sep 24, 2024
Copy link
Member

@mvitale1989 mvitale1989 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the changes Rahim, lgtm overall! I have left a few comments with some suggestions.

Also I was looking up to check if the throttling could be done directly with mitmproxy, instead of changing the host network config, and found your comment in the related issue, which was awesome 😄 🙌

.github/actions/bandwidth-throttling/action.yml Outdated Show resolved Hide resolved
.github/workflows/e2e-detox-pr.yml Show resolved Hide resolved
.github/workflows/e2e-detox-pr.yml Outdated Show resolved Hide resolved
.github/workflows/e2e-detox-pr.yml Outdated Show resolved Hide resolved
.github/actions/start-proxy/action.yml Show resolved Hide resolved
@yasserfaraazkhan yasserfaraazkhan added the E2E iOS tests for PR Run iOS E2E Detox tests label Sep 27, 2024
@github-actions github-actions bot removed the E2E iOS tests for PR Run iOS E2E Detox tests label Sep 27, 2024
Copy link
Contributor

@yasserfaraazkhan yasserfaraazkhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@rahimrahman rahimrahman added E2E Android tests for PR Run Android E2E Detox tests E2E iOS tests for PR (LBW 1) Run iOS E2E Detox tests on Low Bandwidth and removed E2E Android tests for PR Run Android E2E Detox tests labels Oct 5, 2024
@github-actions github-actions bot removed the E2E iOS tests for PR (LBW 1) Run iOS E2E Detox tests on Low Bandwidth label Oct 5, 2024
@rahimrahman rahimrahman added E2E iOS tests for PR Run iOS E2E Detox tests E2E iOS tests for PR (LBW 1) Run iOS E2E Detox tests on Low Bandwidth labels Oct 5, 2024
@github-actions github-actions bot removed E2E iOS tests for PR Run iOS E2E Detox tests E2E iOS tests for PR (LBW 1) Run iOS E2E Detox tests on Low Bandwidth labels Oct 5, 2024
@rahimrahman rahimrahman added E2E iOS tests for PR Run iOS E2E Detox tests 4: Reviews Complete All reviewers have approved the pull request and removed 4: Reviews Complete All reviewers have approved the pull request E2E iOS tests for PR Run iOS E2E Detox tests labels Oct 6, 2024
Copy link
Member

@mvitale1989 mvitale1989 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Rahim, the only needed change to avoid a confusing users would be around adding the label name to the concurrency group, please check this comment

@rahimrahman
Copy link
Contributor Author

@mvitale1989 done! Thanks so much for all your suggestions and feedback!

@mvitale1989 mvitale1989 self-requested a review October 14, 2024 06:54
@rahimrahman rahimrahman merged commit e7d597f into main Oct 14, 2024
4 checks passed
@rahimrahman rahimrahman deleted the test-random-github-action-experiments branch October 14, 2024 13:56
@amyblais amyblais added this to the v2.23.0 milestone Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2: Dev Review Requires review by a core commiter release-note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants