Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Cloud Storage whitelist buckets by regex #85

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

damiano1996
Copy link

Google Cloud Storage whitelist buckets by regex

Description

This pull request adds a new feature to Google Cloud Storage plugin that allows for the whitelisting of buckets using regular expressions. This feature can be particularly useful in a data mesh context, where it's common to segregate data domains and products based on specific naming conventions.

Before this change, whitelisting buckets had to be done manually, which was a time-consuming and error-prone task. With this new feature, users can now choose to whitelist buckets by listing their names individually or by defining a regular expression to match multiple buckets simultaneously. This makes the process of whitelisting buckets more efficient and less prone to errors, which is especially important in a data mesh context where data quality and governance are critical.

Examples

Here are some examples of how this new feature could be used:

  • Whitelist all buckets that contain the word "sales": ^sales-.*
  • Whitelist all buckets that belong to a specific domain: ^mydomain-.*
  • Whitelist all buckets that start with a specific date: ^2023-03-.*

Screenshots

The screenshot below shows the classic option:

dremio_01_list

This one shows the new option with the regex:

dremio_02_regex.

@pavloseroAgile
Copy link

pavloseroAgile commented Jul 9, 2024

It looks very useful, we really need this feature!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants