[WIP] Feature/buisness type database changes #15154

cccs-RyanS · 2021-06-14T18:59:09Z

SUMMARY

Our users are leveraging Superset to explore and visualize network security logs. These logs often contain IP, port, hostname, and/or protocol fields, among others. While such fields might be stored as simple INTEGERS or VARCHARS in the backend database, these fields also have a “business” type which, if leveraged, can provide additional context and improve the user experience.

The following code changes are the first steps towards adding these types currently they are simply added text fields

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Has associated issue:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

github-actions · 2021-07-12T07:01:03Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

betodealmeida · 2021-07-14T20:14:32Z

superset/connectors/base/models.py

@@ -533,6 +533,7 @@ class BaseColumn(AuditMixinNullable, ImportExportMixin):
    verbose_name = Column(String(1024))
    is_active = Column(Boolean, default=True)
    type = Column(String(32))
+    business_type = Column(String(255))


Hi, @cccs-RyanS... do you want to give more context about this PR before putting work into it? I'm worried that this will need a DB migration, and this columns seems very specific.

Hey so my org had discussed this with Ville Brofeldt and I was waiting on some feedback (which is why its still a draft with very little context, my apologies). The idea was to have business types on top of the db types to have added flexibility. Let me speak to my coworker and I can get you a bunch of extra context and I will add more context to the PR. Thanks for taking a look!

github-actions · 2021-07-15T20:39:43Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-07-15T23:23:48Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-07-20T00:42:43Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-07-21T00:25:43Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-07-21T18:17:51Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-07-21T20:55:33Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-07-22T22:37:19Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-08-02T19:46:39Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-08-03T16:11:23Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-08-04T19:13:31Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-08-05T15:35:17Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-08-06T17:06:09Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-08-10T16:30:51Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-08-11T00:52:49Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-08-17T23:28:16Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-09-09T16:47:29Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-09-22T22:10:29Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-09-23T08:29:18Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-09-27T08:26:38Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-10-13T04:41:25Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-10-25T10:54:26Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-10-29T16:34:18Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-11-14T17:57:51Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-11-24T11:43:36Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-11-26T10:29:16Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-12-03T18:29:14Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-12-13T21:05:33Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-12-13T22:29:10Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-12-15T09:19:24Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-12-16T00:17:37Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-12-21T15:59:54Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2021-12-22T20:17:10Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-01-05T02:18:19Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-01-12T17:26:10Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-01-24T19:04:04Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-02-08T08:19:58Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-02-24T19:03:11Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-02-25T18:51:51Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-03-02T18:21:00Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-03-03T20:18:08Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-03-04T00:58:04Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

github-actions · 2022-03-04T16:50:50Z

⚠️ @cccs-RyanS Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

villebro · 2022-03-07T09:40:24Z

Closing in favor of #18794

cccs-RyanS added 2 commits June 14, 2021 18:33

[cherry-pick] Adding business type to models

c90ea76

Running linter

f33d620

cccs-RyanS requested a review from a team as a code owner June 14, 2021 18:59

pull-request-size bot added the size/M label Jun 14, 2021

cccs-RyanS marked this pull request as draft June 14, 2021 19:00

Merge with master

a2a802e

pull-request-size bot added size/L and removed size/M labels Jul 6, 2021

adding deature flage ENABLE_BUSINESS_TYPES

306c77f

betodealmeida self-requested a review July 12, 2021 15:01

Adding feature flag

c0eef48

betodealmeida requested changes Jul 14, 2021

View reviewed changes

cccs-rc mentioned this pull request Jul 14, 2021

[SIP-68] A better model for Datasets #14909

Closed

betodealmeida added the risk:db-migration PRs that require a DB migration label Nov 14, 2021

villebro closed this Mar 7, 2022

[WIP] Feature/buisness type database changes #15154

[WIP] Feature/buisness type database changes #15154

Conversation

cccs-RyanS commented Jun 14, 2021 • edited Loading

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

github-actions bot commented Jul 12, 2021

betodealmeida Jul 14, 2021

Choose a reason for hiding this comment

cccs-RyanS Jul 15, 2021 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Jul 15, 2021

github-actions bot commented Jul 15, 2021

github-actions bot commented Jul 20, 2021

github-actions bot commented Jul 21, 2021

github-actions bot commented Jul 21, 2021

github-actions bot commented Jul 21, 2021

github-actions bot commented Jul 22, 2021

github-actions bot commented Aug 2, 2021

github-actions bot commented Aug 3, 2021

github-actions bot commented Aug 4, 2021

github-actions bot commented Aug 5, 2021

github-actions bot commented Aug 6, 2021

github-actions bot commented Aug 10, 2021

github-actions bot commented Aug 11, 2021

github-actions bot commented Aug 17, 2021

github-actions bot commented Sep 9, 2021

github-actions bot commented Sep 22, 2021

github-actions bot commented Sep 23, 2021

github-actions bot commented Sep 27, 2021

github-actions bot commented Oct 13, 2021

github-actions bot commented Oct 25, 2021

github-actions bot commented Oct 29, 2021

github-actions bot commented Nov 14, 2021

github-actions bot commented Nov 24, 2021

github-actions bot commented Nov 26, 2021

github-actions bot commented Dec 3, 2021

github-actions bot commented Dec 13, 2021

github-actions bot commented Dec 13, 2021

github-actions bot commented Dec 15, 2021

github-actions bot commented Dec 16, 2021

github-actions bot commented Dec 21, 2021

github-actions bot commented Dec 22, 2021

github-actions bot commented Jan 5, 2022

github-actions bot commented Jan 12, 2022

github-actions bot commented Jan 24, 2022

github-actions bot commented Feb 8, 2022

github-actions bot commented Feb 24, 2022

github-actions bot commented Feb 25, 2022

github-actions bot commented Mar 2, 2022

github-actions bot commented Mar 3, 2022

github-actions bot commented Mar 4, 2022

github-actions bot commented Mar 4, 2022

villebro commented Mar 7, 2022

cccs-RyanS commented Jun 14, 2021 •

edited

Loading

cccs-RyanS Jul 15, 2021 •

edited

Loading