Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run benchmark script in CI #844

Merged
merged 24 commits into from
Sep 27, 2022
Merged

Run benchmark script in CI #844

merged 24 commits into from
Sep 27, 2022

Conversation

pankajastro
Copy link
Contributor

@pankajastro pankajastro commented Sep 14, 2022 •
edited
Loading

closes: #443

  • Add a scheduled workflow to run the benchmark script at 6hrs intervals for initial observation later will change it to run on a weekly basis
  • Removed teardown for now

@codecov
Copy link

codecov bot commented Sep 16, 2022 •
edited
Loading

Codecov Report

Base: 94.27% // Head: 94.27% // No change to project coverage 👍

Coverage data is based on head (fed843b) compared to base (b5b1f88).
Patch has no changes to coverable lines.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main     #844   +/-   ##
=======================================
  Coverage   94.27%   94.27%           
=======================================
  Files          47       47           
  Lines        2097     2097           
  Branches      267      267           
=======================================
  Hits         1977     1977           
  Misses         85       85           
  Partials       35       35

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@pankajastro pankajastro force-pushed the benchmark_ci branch 5 times, most recently from 72bd260 to 3056e23 Compare September 19, 2022 12:19
@pankajastro pankajastro changed the title [WIP] Test workflow config [WIP] Run benchmark script in CI Sep 20, 2022
@pankajastro pankajastro force-pushed the benchmark_ci branch 5 times, most recently from dcd3a21 to f3d0a7c Compare September 20, 2022 18:20
@pankajastro pankajastro changed the title [WIP] Run benchmark script in CI Run benchmark script in CI Sep 21, 2022
@pankajastro pankajastro marked this pull request as ready for review September 21, 2022 04:19
tatiana
tatiana reviewed Sep 21, 2022
View reviewed changes
Copy link
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pankajastro this will help us with regression tests!

Do you have a link to an successful job where we run this Github action workflow? Did you have a chance of confirming the correspondent GCS file & bigquery were set as expected?

@pankajastro
Copy link
Contributor Author

pankajastro commented Sep 21, 2022 •
edited
Loading

@pankajastro this will help us with regression tests!

Do you have a link to an successful job where we run this Github action workflow? Did you have a chance of confirming the correspondent GCS file & bigquery were set as expected?

this job succeeded https://github.com/astronomer/astro-sdk/actions/runs/3092321371/jobs/5011687626 which is corresponding to commit 7884fcad6b5d53d10323aee724dc68d8b9096a2e but I'm not able to see the result I checked with SELECT * FROM `astronomer-dag-authoring.benchmark.load_files_to_database` ORDER BY execution_date DESC

@pankajastro pankajastro marked this pull request as draft September 21, 2022 10:58
@pankajastro pankajastro force-pushed the benchmark_ci branch 2 times, most recently from 4b0f5d9 to d3fa29a Compare September 26, 2022 08:59
tatiana
tatiana approved these changes Sep 27, 2022
View reviewed changes
Copy link
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks a lot, @pankajastro !

@pankajastro pankajastro merged commit 753ec2e into main Sep 27, 2022
@pankajastro pankajastro deleted the benchmark_ci branch September 27, 2022 18:34
utkarsharma2 pushed a commit that referenced this pull request Oct 4, 2022
@pankajastro @utkarsharma2
Run benchmark script in CI (#844)
391c33d 
closes: #443 

- Add a scheduled workflow to run the benchmark script at 6hrs intervals
for initial observation later will change it to run on a weekly basis
- Removed teardown for now
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@feluelle feluelle feluelle left review comments

@utkarsharma2 utkarsharma2 utkarsharma2 left review comments

@tatiana tatiana tatiana approved these changes

@kaxil kaxil Awaiting requested review from kaxil

@dimberman dimberman Awaiting requested review from dimberman dimberman is a code owner

@sunank200 sunank200 Awaiting requested review from sunank200 sunank200 is a code owner

@pankajkoti pankajkoti Awaiting requested review from pankajkoti

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Set the CI to run and publish benchmark reports once a week
4 participants
@pankajastro @tatiana @utkarsharma2 @feluelle