[security] Refactor security code into SupersetSecurityManager #4565

timifasubaa · 2018-03-07T19:00:34Z

This PR refactors all the security methods into the extensible SupersetSecurityManager. I left has_access in utils.py for now as it is defined differently from the method with the same name in fab's security manager. I will make that change in another PR.

Historically, superset defined its security methods in different files. All over the code base, there are calls to security methods in security.py, utils.py, fab's security manager and the base superset view. By consolidating all the logic in one place it creates one abstraction for all security in superset and makes the code more understandable.
One example of a change resulting from this consolidation of the security logic is
security.merge_perm(sm, permission_name, view_menu_name)
becoming
sm.merge_perm(permission_name, view_menu_name)
This is much cleaner in several ways as a security object is not receiving another security object as a parameter. There are other examples of methods in utils directly accessing fab security manager's private methods.

@john-bodley @mistercrunch

codecov-io · 2018-03-15T18:24:49Z

Codecov Report

Merging #4565 into master will decrease coverage by 0.02%.
The diff coverage is 70.95%.

@@            Coverage Diff            @@
##           master   #4565      +/-   ##
=========================================
- Coverage   71.52%   71.5%   -0.03%     
=========================================
  Files         190     190              
  Lines       14927   14935       +8     
  Branches     1102    1102              
=========================================
+ Hits        10677   10679       +2     
- Misses       4247    4253       +6     
  Partials        3       3

Impacted Files	Coverage Δ
superset/connectors/druid/models.py	`78.85% <100%> (ø)`	⬆️
superset/connectors/sqla/models.py	`77.91% <100%> (ø)`	⬆️
superset/utils.py	`88.12% <100%> (+0.15%)`	⬆️
superset/data/__init__.py	`100% <100%> (ø)`	⬆️
superset/models/helpers.py	`87.5% <100%> (ø)`	⬆️
superset/models/sql_lab.py	`98.59% <100%> (ø)`	⬆️
superset/connectors/druid/views.py	`68.02% <16.66%> (ø)`	⬆️
superset/views/base.py	`67.12% <20%> (+8.86%)`	⬆️
superset/connectors/sqla/views.py	`71.56% <20%> (ø)`	⬆️
superset/cli.py	`47.74% <33.33%> (-0.65%)`	⬇️
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9d85bd...c766500. Read the comment docs.

timifasubaa · 2018-03-21T21:05:40Z

ping

mistercrunch · 2018-03-21T23:20:12Z

Why 2 security managers? To me the inheritance scheme should be SupersetSecurityManager (new abstraction) inherits from FAB's SecurityManager, and environment-specific say AirbnbSupersetSecurityManager should be forced to derived SupersetSecurityManager.

I don't think it makes sense to grow in the direction of having many security managers. What's next? DatabaseSecurityManager, DashboardSecurityManager?

The scheme I described is a [necessary] breaking change since I'd assume in most environment have a custom SecurityManager that currently derives FAB's and will need to change to derive Superset's.

An [perhaps compatible] alternative would be a SupersetSecurityManagerMixin that we'd somehow inject into FAB's or the enviornment-specific SecurityManger. Not sure if runtime injection of mixin is an acceptable pattern though.

john-bodley · 2018-03-23T17:12:30Z

@mistercrunch the thinking was that the default security manager is still tightly coupled with FAB concepts (views/menus) which correspond to security related to the app UI and API. These concepts are irrelevant from a data security perspective and thus the reason for two security managers.

mistercrunch

This looks great.

After reading the PR I think it should be in scope to rename sm to security_manager everywhere. Rename all the things!

Curious whether you ran into circular deps issues?

mistercrunch · 2018-03-27T16:03:10Z

superset/__init__.py

+custom_sm = app.config.get('CUSTOM_SECURITY_MANAGER')
+if not custom_sm:
+    custom_sm = SupersetSecurityManager
+


We should absolutely raise with an explicit message with perhaps a link to the UPDATE notes if not issubclass(custom_sm, SupersetSecurityManager)

mistercrunch · 2018-03-27T16:05:26Z

superset/cli.py

@@ -16,7 +16,7 @@
 from pathlib2 import Path
 import yaml

-from superset import app, db, dict_import_export_util, security, utils
+from superset import app, data, db, dict_import_export_util, sm, utils


This could be out-of-scope for this PR, but it would be nice to always be explicit about sm and always call it security_manager (my bad!).

It may be reasonable to take it on as part of this PR.

I don’t mind the shortened sm (inline with db and other variables) as it’s used frequently and is considerably shorter than security_manager (2 characters vs. 16).

mistercrunch · 2018-03-27T16:07:13Z

superset/connectors/sqla/views.py

@@ -13,7 +13,7 @@
 from flask_babel import lazy_gettext as _
 from past.builtins import basestring

-from superset import appbuilder, db, security, sm, utils
+from superset import appbuilder, db, sm, utils


Nice! Having both security and sm was confusing, we should have a single entry point for all security checks and operations.

mistercrunch · 2018-03-27T16:09:05Z

superset/__init__.py

@@ -149,12 +150,16 @@ def index(self):
        return redirect('/superset/welcome')


+custom_sm = app.config.get('CUSTOM_SECURITY_MANAGER')


Then you could sm = app.config.get('CUSTOM_SECURITY_MANAGER') or SupersetSecurityManager :)

timifasubaa · 2018-03-27T18:30:54Z

@mistercrunch @john-bodley Done!

john-bodley · 2018-03-27T20:47:24Z

superset/__init__.py

+    raise Exception(
+        """Your CUSTOM_SECURITY_MANAGER must now extend SupersetSecurityManager,
+         not FAB's security manager.
+         See https://github.com/apache/incubator-superset/pull/4565""")


I'm not certain whether we should be referencing other PRs in the code. Given this is a breaking PR you may want to follow this pattern: #4587.

john-bodley · 2018-03-27T20:47:57Z

superset/cli.py

@@ -28,7 +28,8 @@
 @manager.command
 def init():
    """Inits the Superset application"""
-    security.sync_role_definitions()
+    utils.get_or_create_main_db()


Why is this now needed?

Previously it was called inside of sync_role_definitions and that led to weird cyclical dependencies after moving the method to utils.py.
This also make it clear what steps it takes to initialize rather than bundle them together inside one method.

timifasubaa

I added an updating.md as suggested.

timifasubaa · 2018-03-27T20:54:23Z

superset/cli.py

@@ -28,7 +28,8 @@
 @manager.command
 def init():
    """Inits the Superset application"""
-    security.sync_role_definitions()
+    utils.get_or_create_main_db()


Previously it was called inside of sync_role_definitions and that led to weird cyclical dependencies after moving the method to utils.py.
This also make it clear what steps it takes to initialize rather than bundle them together inside one method.

…e#4565) * move access permissions methods to security manager * consolidate all security methods into SupersetSecurityManager * update security method calls * update calls from tests * move get_or_create_main_db to utils * raise if supersetsecuritymanager is not extended * rename sm to security_manager

feng-tao · 2018-04-03T21:22:33Z

superset/security.py

+        def merge_pv(view_menu, perm):
+            """Create permission view menu only if it doesn't exist"""
+            if view_menu and perm and (view_menu, perm) not in all_pvs:
+                self.merge_perm(view_menu, perm)


should it be merge_perm(perm, view_menu) as the definition is " def merge_perm(self, permission_name, view_menu_name):" ? @timifasubaa @mistercrunch (#4745)

…e#4565) * move access permissions methods to security manager * consolidate all security methods into SupersetSecurityManager * update security method calls * update calls from tests * move get_or_create_main_db to utils * raise if supersetsecuritymanager is not extended * rename sm to security_manager

timifasubaa force-pushed the move_access_permissions_to_security_manager branch 13 times, most recently from 69ab53d to 071a823 Compare March 12, 2018 08:42

timifasubaa force-pushed the move_access_permissions_to_security_manager branch 5 times, most recently from fd9a753 to ba3009f Compare March 15, 2018 18:07

timifasubaa force-pushed the move_access_permissions_to_security_manager branch 4 times, most recently from dda246b to 56d0116 Compare March 16, 2018 06:12

timifasubaa changed the title ~~[WIP] Move access permissions methods to security manager~~ Move access permissions methods to security manager Mar 16, 2018

timifasubaa force-pushed the move_access_permissions_to_security_manager branch 3 times, most recently from 4222e46 to 2ff5790 Compare March 25, 2018 07:32

timifasubaa force-pushed the move_access_permissions_to_security_manager branch from c2e7996 to 940e71b Compare March 27, 2018 02:37

move get_or_create_main_db to utils

a385444

timifasubaa force-pushed the move_access_permissions_to_security_manager branch from 940e71b to a385444 Compare March 27, 2018 02:40

mistercrunch requested changes Mar 27, 2018

View reviewed changes

raise if supersetsecuritymanager is not extended

4124ac0

timifasubaa force-pushed the move_access_permissions_to_security_manager branch 5 times, most recently from 89053da to 6903ba1 Compare March 27, 2018 18:01

john-bodley reviewed Mar 27, 2018

View reviewed changes

timifasubaa force-pushed the move_access_permissions_to_security_manager branch from 6903ba1 to 88d3582 Compare March 27, 2018 22:05

rename sm to security_manager

c766500

timifasubaa force-pushed the move_access_permissions_to_security_manager branch from 88d3582 to c766500 Compare March 27, 2018 22:08

timifasubaa commented Mar 27, 2018

View reviewed changes

john-bodley approved these changes Mar 27, 2018

View reviewed changes

mistercrunch approved these changes Mar 27, 2018

View reviewed changes

john-bodley merged commit 8dd052d into apache:master Mar 27, 2018

timifasubaa mentioned this pull request Mar 28, 2018

Add permissions decider for delegating access controls. #4409

Closed

feng-tao reviewed Apr 3, 2018

View reviewed changes

maver1ck mentioned this pull request May 29, 2018

Superset issue #4354 - Properly filter datasources #4355

Closed

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.25.0 labels Feb 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[security] Refactor security code into SupersetSecurityManager #4565

[security] Refactor security code into SupersetSecurityManager #4565

timifasubaa commented Mar 7, 2018 •

edited

Loading

codecov-io commented Mar 15, 2018 •

edited

Loading

timifasubaa commented Mar 21, 2018

mistercrunch commented Mar 21, 2018

john-bodley commented Mar 23, 2018

mistercrunch left a comment

mistercrunch Mar 27, 2018

mistercrunch Mar 27, 2018

john-bodley Mar 27, 2018

mistercrunch Mar 27, 2018

mistercrunch Mar 27, 2018

timifasubaa commented Mar 27, 2018

john-bodley Mar 27, 2018 •

edited

Loading

john-bodley Mar 27, 2018

timifasubaa Mar 27, 2018

timifasubaa left a comment

timifasubaa Mar 27, 2018

feng-tao Apr 3, 2018

		@@ -149,12 +150,16 @@ def index(self):
		return redirect('/superset/welcome')


		custom_sm = app.config.get('CUSTOM_SECURITY_MANAGER')

[security] Refactor security code into SupersetSecurityManager #4565

[security] Refactor security code into SupersetSecurityManager #4565

Conversation

timifasubaa commented Mar 7, 2018 • edited Loading

codecov-io commented Mar 15, 2018 • edited Loading

Codecov Report

timifasubaa commented Mar 21, 2018

mistercrunch commented Mar 21, 2018

john-bodley commented Mar 23, 2018

mistercrunch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timifasubaa commented Mar 27, 2018

john-bodley Mar 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timifasubaa left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timifasubaa commented Mar 7, 2018 •

edited

Loading

codecov-io commented Mar 15, 2018 •

edited

Loading

john-bodley Mar 27, 2018 •

edited

Loading