[SIP-3] Scheduled email reports for Slices / Dashboards #5294

mahendra · 2018-06-27T00:00:04Z

Motivation

Scheduled email reports (of dashboards and slices) has been a long standing ask from the superset community. The feature also provides superset a competitive advantage over tools like Looker.

Proposed change

Scheduled email reports for slice and dashboard visualization (Attachment or inline)
Scheduled email reports for slice data (CSV attachment on inline table)
Each schedule has a list of recipients (all of them can receive a single mail, or separate mails)
All outgoing mails can have a mandatory bcc - for audit purposes.
Each dashboard/slice can have multiple schedules.

How does this work.

Users specify a crontab schedule for a slice / dashboard.
We use celerybeat to schedule tasks for delivering reports every hour.
Each celery task uses selenium (firefox/chrome) webdriver to crawl the actual dashboard (including visualizations) and mail it out to the recipients.

New dependencies

Python libraries
- selenium
- croniter
Firefox webdriver installable for your OS

New or Changed Public Interfaces

The main menu adds two new entries. This will be organized better in a future PR for categorizing dashboards and slices.

Migration plan

Requires a db migration - handled automatically via alembic

Rejected alternatives

Considered running PhantomJS, but selenium is deprecating support for it
Considered embedding image URLs within the email, but it can create issues where superset is hosted within corporate VPNs.

Future work

Better schedule management - currently the beat job will (re)-schedule all the jobs for an hour if it is re-run. We can get better at this.
Better visualization of configured schedules for each dashboard / slice (coming soon in another PR)
Better UI for scheduling reports (coming soon)
Better menu placement (coming soon as part of another PR)
Alternate report delivery mechanisms, if needed.
Try using an existing webdriver instance for all jobs. (to save on memory/cpu)

Screenshots

Admin

Menus

Dashboard schedules

Slice schedules

Delivery

Dashboard via email

Slice via email (inline)

Slice data email (inline)

Slice data as CSV attachment

FAQ's

Why did we not use phantomjs?

(I can make the change if anyone needs/prefers phantomjs).

phantomjs had the following issues

The feature was originally developed using phantomjs. However, selenium emitted a deprecation notice for phantomjs, so I stopped working with it.
Phantomjs does not support taking screenshots of elements (only the entire page). Firefox is the only browser which supports this.
In my testing, I noticed that rendering quality was not as good in phantomjs (fonts). Chrome > Firefox > PhantomJS (did not investigate further)

codecov-io · 2018-06-30T19:34:18Z

Codecov Report

Merging #5294 into master will increase coverage by 0.16%.
The diff coverage is 77.46%.

@@            Coverage Diff             @@
##           master    #5294      +/-   ##
==========================================
+ Coverage   73.31%   73.47%   +0.16%     
==========================================
  Files          67       72       +5     
  Lines        9589     9979     +390     
==========================================
+ Hits         7030     7332     +302     
- Misses       2559     2647      +88

Impacted Files	Coverage Δ
superset/views/core.py	`74.96% <ø> (ø)`	⬆️
superset/tasks/__init__.py	`100% <100%> (ø)`
superset/models/__init__.py	`100% <100%> (ø)`	⬆️
superset/config.py	`93.42% <100%> (+1.05%)`	⬆️
superset/tasks/celery_app.py	`100% <100%> (ø)`
superset/views/__init__.py	`100% <100%> (ø)`	⬆️
superset/sql_lab.py	`71.34% <100%> (ø)`	⬆️
superset/views/schedules.py	`59.09% <59.09%> (ø)`
superset/tasks/schedules.py	`80.1% <80.1%> (ø)`
superset/models/schedules.py	`90.9% <90.9%> (ø)`
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4579b12...8886068. Read the comment docs.

mahendra · 2018-06-30T22:13:53Z

@mistercrunch I had tagged this PR as [WIP], but I now have finished all the functionality, test cases, lint, flake8 etc.

Is there a group / set of reviewers I should ping for review?

The py27-mysql test seems to be flaky ... I saw that it has failed for other PRs as well.

mahendra · 2018-07-03T05:07:40Z

@mistercrunch thank you for fixing the flaky test.

mistercrunch

Did a quick first pass on reading the code, looks high quality. It's a large feature and will require thorough review as committers will have to maintain this moving forward.

I'm thinking we should have a feature flag to allow hiding the feature if it's not configured properly.

mistercrunch · 2018-07-04T23:31:48Z

superset/config.py

+    )
+    CELERY_RESULT_BACKEND = 'db+sqlite:///celery_results.sqlite'
+    CELERYD_LOG_LEVEL = 'DEBUG'
+    CELERYD_PREFETCH_MULTIPLIER = 10


Why 10? 1 may be better. Personally I prefer leaving the messages in the queue instead of prefetching on a potentially busy worker

Agree on 1 - I believe this was some of my testing code which made it's way through.

mistercrunch · 2018-07-04T23:47:33Z

superset/models/schedules.py

+    elif report_type == ScheduleType.slice.value:
+        return SliceEmailSchedule
+
+    return None


that's implicit in python, no need for this line

mahendra · 2018-07-05T00:03:07Z

@mistercrunch thank you for the review. I agree on hiding it behind a configuration. Shall I introduce a global config variable ENABLE_SCHEDULED_EMAIL_REPORTS (False by default)

If disabled

celery beat jobs will not get scheduled
the option for scheduling reports will disappear from the menus

Please do let me know and I will make the changes.

As regards to maintenance, I will continue to support it. This feature is important for my company (https://github.com/Affirm/) and we will rely heavily on it. I am also planning to have my team start adding more features to superset soon.

mistercrunch · 2018-07-05T00:05:07Z

Haven't done a very thorough review yet, but wanted to mention a slightly different approach and discuss pro/cons.

I always though we'd add endpoints that would build an image on the fly and return it (say superset/chart_image/{chart_id}.png, superset/dashboard_image/{dash_id}.png), and an email feature would simply build an html email with an tag pointing to it.

There are some important security considerations. First the approach requires auth + perms, which may not work on mobile when Superset is on intranet, and no vpn is setup on the phone. It has the advantage of being more secure in many ways, though it may just not work in many envs (mobile is not on VPN). Forwarding an email would only work for the recipient if she/he has access to the chart for instance.

The binary approach has a lot of advantages too, it will just work.

Maybe we need both? Maybe environments would define which method they allow? Maybe the schedule would allow you to pick which method to use in that particular case?

mahendra · 2018-07-05T00:16:08Z

@mistercrunch I did consider the idea of URLs for the image and embedding it within mails. However, I decided against it for the following reasons

Generating the image (using a webdriver) was an expensive operation (CPU/Memory). I thought it would be better to generate it once in the background and send it off to multiple recipients in one shot. We could cache the URLs and reduce the hit on the servers.
As you noted - VPN limitations. We have such a setup, but we have a large mobile sales team which will appreciate full delivery of the data for quick reference. (mobile device VPN is a pain)
We also plan to send out mails to external entities, who will not have access to our mobile setup.

I however do like the embedded URL approach to prevent accidental forwards of internal reports to external folks :-)

I agree on having a both approaches. The current approach is easy to extend.

The celery jobs generate the images and store/cache them (for a period of time)
The emails contain the links or blobs (based on a config)
If someone hits the URL and the image does not exist / has expired, we fire off a celery job and generate it.

I can work on this in a future ticket (this issue - #623)

mahendra · 2018-07-05T00:50:55Z

superset/config.py

@@ -446,6 +446,9 @@ class CeleryConfig(object):
 PLANNED_V2_AUTO_CONVERT_DATE = None  # e.g. '2018-06-16'
 V2_FEEDBACK_URL = None  # e.g., 'https://goo.gl/forms/...'

+# Enable / disable scheduled email reports


@mistercrunch config to hide the feature.

mistercrunch · 2018-07-06T14:56:29Z

Cool. We'll take the time to review the current feature while keeping in mind we'll eventually support both.

YHGui · 2018-07-18T08:38:28Z

@mahendra hi, I'd like to try the feature about email with your code. can I get the source code and deploy it on my machine? Thanks

mahendra · 2018-07-18T16:27:48Z

Hi @YHGui, Which future feature do you want to see the code for?

YHGui · 2018-07-19T01:16:55Z

@mahendra soryy, it's a typo. I'd like to try the email feature.

mahendra · 2018-07-19T01:20:32Z

@YHGui - oh, you want to try this PR on your machine? The branch lives here - https://github.com/Affirm/incubator-superset/tree/scheduled

YHGui · 2018-07-19T01:22:22Z

@mahendra Thanks a lot! would you please tell me your email for some questions?
after I merged the code with source code of version 0.26.3, deployed it on my machine, get the error "no such table: dashboard_email_schedules", and I run the command "Superset db upgrade", but it didn't work and get the same errors, how to solve the problem? Thanks!

I cloned your branch and deploy on my machine, with my Python environment, I changed some function in urllib for Python2.7 and make "ENABLE_SCHEDULED_EMAIL_REPORTS" True, make the "EMAIL_REPORTS_WEBDRIVER" chrome, the "SMTP_HOST" is 'smtp.gmail.com' and port is 587, after set the crontab '*/3 * * * *' but I cannot get report from Superset, did I miss some details? Thanks!

mahendra · 2018-07-25T05:08:33Z

@YHGui apologies for the delay here. Two things to do:

no such table: dashboard_email_schedules - looks like the alembic migration is not run correctly.
About not receiving the emails, you need to follow the documentation in my PR and install selenium and chrome webdriver. Then you need to run celery worker and celery beat

mahendra · 2018-07-25T05:09:17Z

@mistercrunch just checking in on this PR. Should I ping any other reviewers?

YHGui · 2018-07-25T06:07:28Z

@mahendra thanks! I did it just like you said above several days ago, install selenium and webdriver, run celery worker and celery beat, after running it, I got time limit exceeded and I change the time limit in config.py, but it doesn't work, I'd like to know some settings, such as EMAIL_REPORTS_USER, does it mean the username who is the owner of the dashboard or admin users?

georgexiong · 2019-03-15T03:38:05Z

@magicansk I pull https://github.com/Affirm/incubator-superset set it up, but i still can not see the schedule/email item on the mange menu. how to get it work?

mahendra · 2019-03-15T06:04:37Z

You need to enable it in configuration. The process is defined in the documentation. Please do let me know if it doesn't work.

…

On Thu, Mar 14, 2019 at 8:39 PM georgexiong ***@***.***> wrote: @magicansk <https://github.com/magicansk> I pull https://github.com/Affirm/incubator-superset set it up, but i still can not see the schedule/email item on the mange menu. how to get it work? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5294 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHFFPfxxMPmPU2yfr1fHAYDXL2io0asks5vWxXrgaJpZM4U42zO> .

-- Mahendra http://twitter.com/mahendra gpg key: E31C89FD <https://pgp.mit.edu/pks/lookup?op=get&search=0x7BEC08A5E31C89FD>

yuguang · 2019-04-05T00:18:52Z

@mahendra I have enabled email reports. I keep getting this error when celery tries to generate an email:

[2019-04-05 00:15:00,319: ERROR/ForkPoolWorker-5] Task email_reports.send[b80dc5
9e-b246-4eeb-81e5-d94b9f1ac4e0] raised unexpected: SessionNotCreatedException('U
nable to find a matching set of capabilities', None, None)
Traceback (most recent call last):
  File "/home/superset/incubator-superset/superset-env/lib/python3.6/site-packag
es/celery/app/trace.py", line 382, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/superset/incubator-superset/superset-env/lib/python3.6/site-packag
es/celery/app/trace.py", line 641, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/superset/apache-superset-0.31.0rc20/superset/tasks/schedules.py",
line 392, in schedule_email_report
    deliver_dashboard(schedule)
  File "/home/superset/apache-superset-0.31.0rc20/superset/tasks/schedules.py",
line 219, in deliver_dashboard
    driver = create_webdriver()
  File "/home/superset/apache-superset-0.31.0rc20/superset/tasks/schedules.py",
line 168, in create_webdriver
    driver = driver_class(**kwargs)
  File "/home/superset/incubator-superset/superset-env/lib/python3.6/site-packag
es/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/home/superset/incubator-superset/superset-env/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/superset/incubator-superset/superset-env/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/superset/incubator-superset/superset-env/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/superset/incubator-superset/superset-env/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities

I have firefox-60.6.1 installed and I am on the release--0.31 branch.

yuguang · 2019-04-05T00:59:59Z

@YHGui Were you able to solve the problem selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities? If so, what were the versions of geckodriver and firefox you used?

starsunny · 2019-05-19T07:47:21Z

@mahendra How can I merge these changes in my existing 0.29 version?

suvarnaC · 2019-06-02T18:45:51Z

@mahendra I tried the email report feature, i am not seeing any errors in the geckodriver.log or nohup for celery worker or superset logs. But i am not getting any emails sent. I checked
send_email_smtp('recipient@gmail.com', 'Hello', 'Test content', app.config)
is sending the email . So my email configuration is working. Where else should i look for issues?

suvarnaC · 2019-06-03T15:56:29Z

i could make it work by changing the firefox options, just added proxy to it.

ganesh8614 · 2019-07-31T13:07:47Z

@YHGui - oh, you want to try this PR on your machine? The branch lives here - https://github.com/Affirm/incubator-superset/tree/scheduled

Hi Mahendra,

I want to implement schedule reports but the above link got expired. Can you please help me on this?

Thanks
Ganesh

ganesh8614 · 2019-07-31T13:18:01Z

@mahendra Thanks a lot! would you please tell me your email for some questions?
after I merged the code with source code of version 0.26.3, deployed it on my machine, get the error "no such table: dashboard_email_schedules", and I run the command "Superset db upgrade", but it didn't work and get the same errors, how to solve the problem? Thanks!

I cloned your branch and deploy on my machine, with my Python environment, I changed some function in urllib for Python2.7 and make "ENABLE_SCHEDULED_EMAIL_REPORTS" True, make the "EMAIL_REPORTS_WEBDRIVER" chrome, the "SMTP_HOST" is 'smtp.gmail.com' and port is 587, after set the crontab '*/3 * * * *' but I cannot get report from Superset, did I miss some details? Thanks!

I want to implement schedule reports Can you please help me on this?

Thanks
Ganesh

ganesh8614 · 2019-08-06T14:48:13Z

@mahendra I tried the email report feature, i am not seeing any errors in the geckodriver.log or nohup for celery worker or superset logs. But i am not getting any emails sent. I checked
send_email_smtp('recipient@gmail.com', 'Hello', 'Test content', app.config)
is sending the email . So my email configuration is working. Where else should i look for issues?

After performing the configuration tried to run the below command and getting the error:

celery worker --app=superset.tasks.celery_app:app --pool=prefork -Ofair -c 4

Error Details******************
_sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: BEGIN IMMEDIATE TRANSACTION]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-08-06 20:15:27,255: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "d:\apache_superset\venv\lib\site-packages\celery\worker\consumer\consumer.py", line 318, in start
blueprint.start(self)
File "d:\apache_superset\venv\lib\site-packages\celery\bootsteps.py", line 119, in start
step.start(parent)
File "d:\apache_superset\venv\lib\site-packages\celery\worker\consumer\consumer.py", line 596, in start
c.loop(*c.loop_args())
File "d:\apache_superset\venv\lib\site-packages\celery\worker\loops.py", line 121, in synloop
connection.drain_events(timeout=2.0)
File "d:\apache_superset\venv\lib\site-packages\kombu\connection.py", line 315, in drain_events
return self.transport.drain_events(self.connection, **kwargs)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 963, in drain_events
get(self._deliver, timeout=timeout)
File "d:\apache_superset\venv\lib\site-packages\kombu\utils\scheduling.py", line 56, in get
return self.fun(resource, callback, **kwargs)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 1001, in _drain_channel
return channel.drain_events(callback=callback, timeout=timeout)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 745, in drain_events
return self._poll(self.cycle, callback, timeout=timeout)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 402, in _poll
return cycle.get(callback)
File "d:\apache_superset\venv\lib\site-packages\kombu\utils\scheduling.py", line 56, in get
return self.fun(resource, callback, **kwargs)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 405, in _get_and_deliver
message = self.get(queue)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\sqlalchemy_init.py", line 96, in _get
self.session.execute('BEGIN IMMEDIATE TRANSACTION')
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\orm\session.py", line 1269, in execute
clause, params or {}
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 988, in execute
return meth(self, multiparams, params)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\sql\elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1107, in _execute_clauseelement
distilled_params,
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\util\compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\util\compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1244, in execute_context
cursor, statement, parameters, context
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: BEGIN IMMEDIATE TRANSACTION]
(Background on this error at: http://sqlalche.me/e/e3q8)

Thanks
Ganesh

saifurrehman · 2019-10-04T22:26:49Z

@mahendra I have configured everything without any errors but still i am not able to see the email option on front end. Can you please help me with it?

yuguang · 2019-10-04T22:51:28Z

Set up a local SMTP debugging server and paste the logs here
https://realpython.com/python-send-email/

prasadbhosale · 2019-10-24T13:20:09Z

@mahendra I have configured everything without any errors but still i am not able to see the email option on front end. Can you please help me with it?

I am also facing this problem.
@mahendra - Pl help us.

starsunny · 2019-10-24T16:45:47Z

Which version are you using? Have you enabled the option in config.py, which is basically for showing the menus in superset UI?

…

On Thu, 24 Oct, 2019, 6:51 PM prasadbhosale, ***@***.***> wrote: @mahendra <https://github.com/mahendra> I have configured everything without any errors but still i am not able to see the email option on front end. Can you please help me with it? I am also facing this problem. @mahendra <https://github.com/mahendra> - Pl help us. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5294>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEZS2ZKATO7JXDL4YF4GHQDQQGOM5ANCNFSM4FHDNTHA> .

toilv · 2019-10-30T01:15:09Z

@mahendra I have configured everything without any errors but still i am not able to see the email option on front end. Can you please help me with it?

I am also facing this problem.
@mahendra - Pl help us.

Yes, i have the same problem. Check connection but no opened connections for SMTP . Superset did not fire any email trigger

nabhosal · 2019-12-03T11:03:35Z

We are using superset with schedule report emailing, it works great for us. In the below article, we have published steps to make deployment easy for superset with schedule report emailing.
https://medium.com/@nileshxbhosale/apache-superset-a-cost-effective-alternative-to-quick-sight-for-data-lake-668868fe51c7

visakhmurukes · 2019-12-24T12:28:29Z

Just tried this feature in Superset. Though I got reports delivered through email as a png image the image didnt have actual report data. It just shows screenshot of report running icon. I guess it got delivered before actual report got rendered. Is the report rendering and email sending process asynchronous? Looks like its not waiting for report to get rendered before sending it through email

shawnz42 · 2020-01-08T03:23:40Z

Just tried this feature in Superset. Though I got reports delivered through email as a png image the image didnt have actual report data. It just shows screenshot of report running icon. I guess it got delivered before actual report got rendered. Is the report rendering and email sending process asynchronous? Looks like its not waiting for report to get rendered before sending it through email

I have the same problem. And I have another question, whythe WEBDRIVER_BASEURL port 8080, not 8088?

sunshinetan · 2020-01-22T09:42:04Z

i used the master branch on win10 and I have configured everything，when i start a Celery worker，it shows RuntimeError: Working outside of application context，can anybody help me pls？

PRamia · 2020-02-11T13:40:53Z

@mahendra I tried the email report feature, i am not seeing any errors in the geckodriver.log or nohup for celery worker or superset logs. But i am not getting any emails sent. I checked
send_email_smtp('recipient@gmail.com', 'Hello', 'Test content', app.config)
is sending the email . So my email configuration is working. Where else should i look for issues?

After performing the configuration tried to run the below command and getting the error:

celery worker --app=superset.tasks.celery_app:app --pool=prefork -Ofair -c 4

Error Details******************
_sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: BEGIN IMMEDIATE TRANSACTION]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-08-06 20:15:27,255: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "d:\apache_superset\venv\lib\site-packages\celery\worker\consumer\consumer.py", line 318, in start
blueprint.start(self)
File "d:\apache_superset\venv\lib\site-packages\celery\bootsteps.py", line 119, in start
step.start(parent)
File "d:\apache_superset\venv\lib\site-packages\celery\worker\consumer\consumer.py", line 596, in start
c.loop(*c.loop_args())
File "d:\apache_superset\venv\lib\site-packages\celery\worker\loops.py", line 121, in synloop
connection.drain_events(timeout=2.0)
File "d:\apache_superset\venv\lib\site-packages\kombu\connection.py", line 315, in drain_events
return self.transport.drain_events(self.connection, **kwargs)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 963, in drain_events
get(self._deliver, timeout=timeout)
File "d:\apache_superset\venv\lib\site-packages\kombu\utils\scheduling.py", line 56, in get
return self.fun(resource, callback, **kwargs)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 1001, in _drain_channel
return channel.drain_events(callback=callback, timeout=timeout)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 745, in drain_events
return self._poll(self.cycle, callback, timeout=timeout)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 402, in _poll
return cycle.get(callback)
File "d:\apache_superset\venv\lib\site-packages\kombu\utils\scheduling.py", line 56, in get
return self.fun(resource, callback, **kwargs)
File "d:\apache_superset\venv\lib\site-packages\kombu\transport\virtual\base.py", line 405, in get_and_deliver
message = self.get(queue) File "d:\apache_superset\venv\lib\site-packages\kombu\transport\sqlalchemy__init.py", line 96, in _get
self.session.execute('BEGIN IMMEDIATE TRANSACTION')
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\orm\session.py", line 1269, in execute
clause, params or {}
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 988, in execute
return meth(self, multiparams, params)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\sql\elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1107, in _execute_clauseelement
distilled_params,
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\util\compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\util\compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1244, in execute_context cursor, statement, parameters, context File "d:\apache_superset\venv\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked [SQL: BEGIN IMMEDIATE TRANSACTION] (Background on this error at: http://sqlalche.me/e/e3q8)

Thanks
Ganesh

i am getting the same error. Could you (or other smart guy) how to fix this issue?
trying hard to get the Emails working but no way.

tong900801 · 2020-06-04T04:24:01Z

Can you share a stey-by-step guide, I didin't get any emails when setting by the document(https://superset.incubator.apache.org/installation.html#email-reports).

Thanks a lot

stevensuting · 2020-07-28T03:48:00Z

Can you share a stey-by-step guide, I didin't get any emails when setting by the document(https://superset.incubator.apache.org/installation.html#email-reports).

Thanks a lot

Step1: cd superset/bin
Step 2: create a file superset_config.py
Step 3: Add the line ENABLE_SCHEDULED_EMAIL_REPORTS = True
Step 4: save the file
Step 5: run superset init
Step 6: run superset run

vinaycd · 2020-11-18T14:40:09Z

@YHGui Were you able to solve the problem selenium.common.exceptions.SessionNotCreatedException: Message: Unable to find a matching set of capabilities? If so, what were the versions of geckodriver and firefox you used?

Can you able to fix this issue?

I am using the following version
geckodriver - 0.24.0
firefox - 68.2.0esr
selenium - 3.141.0

mahendra changed the title ~~Scheduled email reports for Slices / Dashboards~~ [WIP] Scheduled email reports for Slices / Dashboards Jun 27, 2018

mahendra force-pushed the scheduled branch 3 times, most recently from 1e4a0df to 15d1bd9 Compare June 30, 2018 18:17

mahendra changed the title ~~[WIP] Scheduled email reports for Slices / Dashboards~~ Scheduled email reports for Slices / Dashboards Jun 30, 2018

mahendra force-pushed the scheduled branch from 15d1bd9 to 45d4934 Compare June 30, 2018 19:13

mahendra force-pushed the scheduled branch from 45d4934 to 8c65778 Compare June 30, 2018 21:50

mahendra force-pushed the scheduled branch from 8c65778 to bfdb5d2 Compare July 3, 2018 03:30

This was referenced Jul 3, 2018

Support for emailing reports at regular intervals? #1231

Closed

Integrate a headless browser to render Slices as images #1037

Closed

API for getting image #623

Closed

mistercrunch reviewed Jul 4, 2018

View reviewed changes

mahendra force-pushed the scheduled branch from bfdb5d2 to 8efd469 Compare July 5, 2018 00:04

mahendra commented Jul 5, 2018

View reviewed changes

rusackas changed the title ~~[SIP-3] Scheduled email reports for Slices / Dashboards~~ [SIP-03] Scheduled email reports for Slices / Dashboards Dec 8, 2022

pull-request-size bot added the size/XXL label Dec 8, 2022

rusackas changed the title ~~[SIP-03] Scheduled email reports for Slices / Dashboards~~ [SIP-3] Scheduled email reports for Slices / Dashboards Dec 8, 2022

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.34.0 labels Feb 27, 2024

[SIP-3] Scheduled email reports for Slices / Dashboards #5294

[SIP-3] Scheduled email reports for Slices / Dashboards #5294

Conversation

mahendra commented Jun 27, 2018 • edited Loading

Motivation

Proposed change

How does this work.

New dependencies

New or Changed Public Interfaces

Migration plan

Rejected alternatives

Future work

Screenshots

Admin

Menus

Dashboard schedules

Slice schedules

Delivery

Dashboard via email

Slice via email (inline)

Slice data email (inline)

Slice data as CSV attachment

FAQ's

Why did we not use phantomjs?

codecov-io commented Jun 30, 2018 • edited Loading

Codecov Report

mahendra commented Jun 30, 2018

mahendra commented Jul 3, 2018

mistercrunch left a comment

Choose a reason for hiding this comment

mistercrunch Jul 4, 2018 • edited Loading

Choose a reason for hiding this comment

mahendra Jul 4, 2018

Choose a reason for hiding this comment

mistercrunch Jul 4, 2018

Choose a reason for hiding this comment

mahendra Jul 4, 2018

Choose a reason for hiding this comment

mahendra commented Jul 5, 2018

mistercrunch commented Jul 5, 2018

mahendra commented Jul 5, 2018

mahendra Jul 5, 2018

Choose a reason for hiding this comment

mistercrunch commented Jul 6, 2018

YHGui commented Jul 18, 2018 • edited Loading

mahendra commented Jul 18, 2018

YHGui commented Jul 19, 2018

mahendra commented Jul 19, 2018

YHGui commented Jul 19, 2018 • edited Loading

mahendra commented Jul 25, 2018

mahendra commented Jul 25, 2018

YHGui commented Jul 25, 2018

georgexiong commented Mar 15, 2019

mahendra commented Mar 15, 2019 via email

yuguang commented Apr 5, 2019

yuguang commented Apr 5, 2019

starsunny commented May 19, 2019

suvarnaC commented Jun 2, 2019

suvarnaC commented Jun 3, 2019

ganesh8614 commented Jul 31, 2019 • edited Loading

ganesh8614 commented Jul 31, 2019

ganesh8614 commented Aug 6, 2019

saifurrehman commented Oct 4, 2019

yuguang commented Oct 4, 2019

prasadbhosale commented Oct 24, 2019

starsunny commented Oct 24, 2019 via email

toilv commented Oct 30, 2019 • edited Loading

nabhosal commented Dec 3, 2019

visakhmurukes commented Dec 24, 2019

shawnz42 commented Jan 8, 2020 • edited Loading

sunshinetan commented Jan 22, 2020

PRamia commented Feb 11, 2020

tong900801 commented Jun 4, 2020

stevensuting commented Jul 28, 2020

vinaycd commented Nov 18, 2020

mahendra commented Jun 27, 2018 •

edited

Loading

codecov-io commented Jun 30, 2018 •

edited

Loading

mistercrunch Jul 4, 2018 •

edited

Loading

YHGui commented Jul 18, 2018 •

edited

Loading

YHGui commented Jul 19, 2018 •

edited

Loading

ganesh8614 commented Jul 31, 2019 •

edited

Loading

toilv commented Oct 30, 2019 •

edited

Loading

shawnz42 commented Jan 8, 2020 •

edited

Loading