Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: use exception status for logging #21971

Merged
merged 2 commits into from
Nov 14, 2022

Conversation

eschutho
Copy link
Member

@eschutho eschutho commented Oct 29, 2022

SUMMARY

Followup to #21627
In order to be able to log celery errors and warnings separately, I am focusing on report errors for both slack and email and raising either a ReportScheduleClientErrorsException or ReportScheduleSystemErrorsException and then logging according to the status code of each (4xx or 500).
For slack errors, I am also catching each error from the slack api and raising an appropriate http status code, which then helps to inform the logging level for celery.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@eschutho eschutho force-pushed the elizabeth/celery-logs-2 branch 2 times, most recently from 5886783 to d01ef03 Compare October 29, 2022 00:38
@eschutho eschutho changed the title Elizabeth/celery logs 2 DRAFT- refactor: use exception status for logging Oct 29, 2022
@codecov
Copy link

codecov bot commented Oct 29, 2022

Codecov Report

Merging #21971 (357dab2) into master (c870fbe) will increase coverage by 1.42%.
The diff coverage is 75.00%.

❗ Current head 357dab2 differs from pull request most recent head ec6025a. Consider uploading reports for the commit ec6025a to get more accurate results

@@            Coverage Diff             @@
##           master   #21971      +/-   ##
==========================================
+ Coverage   65.56%   66.99%   +1.42%     
==========================================
  Files        1815     1815              
  Lines       69553    69623      +70     
  Branches     7486     7486              
==========================================
+ Hits        45604    46642    +1038     
+ Misses      22015    21047     -968     
  Partials     1934     1934              
Flag Coverage Δ
hive 52.83% <43.75%> (?)
mysql ?
postgres 78.43% <70.31%> (-0.03%) ⬇️
presto 52.73% <43.75%> (?)
python 81.51% <75.00%> (+2.92%) ⬆️
sqlite 76.91% <70.31%> (-0.02%) ⬇️
unit 51.22% <43.75%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/tasks/scheduler.py 79.24% <16.66%> (-2.76%) ⬇️
superset/reports/notifications/slack.py 79.51% <18.18%> (-8.49%) ⬇️
superset/reports/notifications/email.py 98.64% <75.00%> (-1.36%) ⬇️
superset/reports/commands/execute.py 91.55% <94.44%> (-1.06%) ⬇️
superset/errors.py 94.52% <100.00%> (+0.07%) ⬆️
superset/reports/commands/exceptions.py 98.38% <100.00%> (+0.12%) ⬆️
superset/reports/notifications/exceptions.py 100.00% <100.00%> (ø)
superset/utils/log.py 93.91% <100.00%> (+0.21%) ⬆️
superset/db_engine_specs/mysql.py 94.04% <0.00%> (-4.77%) ⬇️
superset/reports/commands/log_prune.py 85.71% <0.00%> (-3.58%) ⬇️
... and 73 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@eschutho eschutho force-pushed the elizabeth/celery-logs-2 branch 3 times, most recently from dd002fc to 357dab2 Compare November 10, 2022 18:28
@eschutho eschutho changed the title DRAFT- refactor: use exception status for logging refactor: use exception status for logging Nov 10, 2022
@eschutho eschutho merged commit ce145c6 into apache:master Nov 14, 2022
@@ -258,6 +261,17 @@ class ReportScheduleStateNotFoundError(CommandException):
message = _("Report Schedule state not found")


class ReportScheduleSystemErrorsException(CommandException, SupersetErrorsException):
errors: List[SupersetError] = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to set an overall status for the array or do we want to do it based upon the most critical error in the error array

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we set the status based upon the most critical



class ReportScheduleClientErrorsException(CommandException, SupersetErrorsException):
status = 400
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we have a status but the exception above doesn't

AnushaErrabelli pushed a commit to preset-io/superset that referenced this pull request Nov 15, 2022
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.1.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset:2022.45 size/L 🚢 2.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants