Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(native-filters): add search all filter options #14710

Merged
merged 12 commits into from
May 24, 2021

Conversation

villebro
Copy link
Member

@villebro villebro commented May 19, 2021

SUMMARY

Add support for "Search all filter options" option to native select filter to obtain feature parity with same functionality in Filter Box. Other changes:

  • Filter Box performed a manual case insensitive comparison for the search all filter functionality by adding the following WHERE clause to the query: lower(${key}) like '%${input}%', which only worked on those backends that support the lower function (a non-ANSI SQL function). This has been replaced by the following logic
  • Byproduct: Add ILIKE operator option to adhoc filter popover (see screenshot)

SCREENSHOT

See below how searching for entries that don't fit into the first 1000 entries are found when the feature is enabled:
https://user-images.githubusercontent.com/33317356/119319233-d88fb880-bc82-11eb-9a11-e63aca9e243f.mp4

Also add case case insensitive LIKE operator to adhoc filter popover to support ILIKE SqlAlchemy operator:
image

Also pull in the following PRs from superset-ui:

TEST PLAN

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@codecov
Copy link

codecov bot commented May 21, 2021

Codecov Report

Merging #14710 (5996b58) into master (bee6f3b) will decrease coverage by 0.01%.
The diff coverage is 66.27%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #14710      +/-   ##
==========================================
- Coverage   77.32%   77.30%   -0.02%     
==========================================
  Files         962      962              
  Lines       49093    49147      +54     
  Branches     6165     6182      +17     
==========================================
+ Hits        37959    37994      +35     
- Misses      10930    10949      +19     
  Partials      204      204              
Flag Coverage Δ
javascript 72.25% <67.46%> (-0.02%) ⬇️
mysql 81.63% <33.33%> (-0.01%) ⬇️
postgres 81.66% <33.33%> (-0.01%) ⬇️
python 81.74% <33.33%> (-0.01%) ⬇️
sqlite 81.26% <33.33%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...tersConfigModal/FiltersConfigForm/DefaultValue.tsx 27.77% <ø> (ø)
...nd/src/dashboard/components/nativeFilters/utils.ts 80.48% <ø> (ø)
...onents/controls/FilterControl/AdhocFilter/index.js 97.01% <ø> (ø)
...tend/src/filters/components/Select/controlPanel.ts 58.33% <0.00%> (-5.31%) ⬇️
...et-frontend/src/filters/components/Select/index.ts 100.00% <ø> (ø)
...nd/src/filters/components/Select/transformProps.ts 71.42% <ø> (ø)
...et-frontend/src/filters/components/Select/types.ts 100.00% <ø> (ø)
superset/connectors/sqla/models.py 88.08% <0.00%> (-0.22%) ⬇️
...veFilters/FilterBar/FilterControls/FilterValue.tsx 67.44% <52.63%> (-0.13%) ⬇️
...c/filters/components/Select/SelectFilterPlugin.tsx 69.23% <60.46%> (-9.35%) ⬇️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bee6f3b...5996b58. Read the comment docs.

@villebro villebro changed the title [WIP] feat(native-filters): add search all filter options feat(native-filters): add search all filter options May 21, 2021
Comment on lines +52 to +84
type DataMaskAction =
| { type: 'ownState'; ownState: JsonObject }
| {
type: 'filterState';
extraFormData: ExtraFormData;
filterState: { value: SelectValue };
};

function reducer(state: DataMask, action: DataMaskAction): DataMask {
switch (action.type) {
case 'ownState':
return {
...state,
ownState: {
...(state.ownState || {}),
...action.ownState,
},
};
case 'filterState':
return {
...state,
extraFormData: action.extraFormData,
filterState: {
...(state.filterState || {}),
...action.filterState,
},
};
default:
return {
...state,
};
}
}
Copy link
Member Author

@villebro villebro May 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to migrate from multiple useStates and useEffects to useReducer, as the full dataMask will be updated by different actions but needs to be submitted as a whole to setDataMask. Therefore, useReducer seemed like a cleaner approach.

Comment on lines 63 to 76
const initSelectValue: SelectValue =
// `defaultValue` can be `FIRST_VALUE` if `defaultToFirstItem` is checked, so need convert it to correct value for Select
defaultValue === FIRST_VALUE ? [] : defaultValue ?? [];

const firstItem: SelectValue = data[0]
? (groupby.map(col => data[0][col]) as string[]) ?? initSelectValue
: initSelectValue;

// If we are in config modal we always need show empty select for `defaultToFirstItem`
const [values, setValues] = useState<SelectValue>(
defaultToFirstItem && appSection !== AppSection.FILTER_CONFIG_MODAL
? firstItem
: initSelectValue,
);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FIRST_VALUE placeholder doesn't really work, as we don't know what the first value is for each individual user, which means that it won't be possible to persist extraFormData at filter configuration time.

@michael-s-molina
Copy link
Member

@villebro I noticed two things:

  • I added a filter of countries. When I opened the filter you can see 'Australia' as one of the values. When I type 'aus', Australia is not shown.
  • When I search for a country and there are no results, there's no way to change my search because the search field is replaced by an empty message
20210522102235847.mp4

@villebro
Copy link
Member Author

villebro commented May 24, 2021

@michael-s-molina this is weird - the "No Results" behavior you describe should not happen after bumping the superset-ui packages. Can you check again that you have the npm bumps applied?

The search problem you faced is due to previously using the LIKE filter type - I have now updated to use ILIKE, which will perform a case insensitive lookup. So now searching for "aus" will find "Australia" on e.g. Postgres.

@michael-s-molina
Copy link
Member

@michael-s-molina this is weird - the "No Results" behavior you describe should not happen after bumping the superset-ui packages. Can you check again that you have the npm bumps applied?

The search problem you faced is due to previously using the LIKE filter type - I have now updated to use ILIKE, which will perform a case insensitive lookup. So now searching for "aus" will find "Australia" on e.g. Postgres.

@villebro After pulling the changes and bumping the superset-ui packages I can confirm that both issues are fixed 😃

Comment on lines +210 to +214
// handle changes coming from application, e.g. "Clear all" button
if (JSON.stringify(values) !== JSON.stringify(filterState.value)) {
handleChange(filterState.value);
}
}, [JSON.stringify(filterState.value)]);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be removed in a follow-up PR when all state handling is moved to the application

import { Select } from 'src/common/components';
import { FIRST_VALUE, PluginFilterSelectProps, SelectValue } from './types';
import { debounce } from 'lodash';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently it's better to import debounce from 'lodash/debounce', as it results in smaller bundle size (and we probably should apply that in other places in superset).
References:
https://stackoverflow.com/questions/35250500/correct-way-to-import-lodash
https://www.blazemeter.com/blog/the-correct-way-to-import-lodash-libraries-a-benchmark

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know 👍

row_limit: 10000,
row_limit: 1000,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default for Filter Box is 1000 rows.

@villebro villebro merged commit e9657af into apache:master May 24, 2021
@villebro villebro deleted the villebro/select-load branch May 24, 2021 13:34
amitmiran137 pushed a commit to nielsen-oss/superset that referenced this pull request May 25, 2021
* master: (163 commits)
  fix(native-filters): Manage default value of filters by superset (apache#14785)
  fix: Additional ResultSet tests (apache#14741)
  chore: added BasicParametersMixin to Redshift (apache#14752)
  fix: make dataset list sort case insensitive (apache#14528)
  fix: use encodeURIComponent when getting table metadata (apache#14790)
  fix: ensure engine is outside parameters (apache#14787)
  database modal should close on connect with tab layout (apache#14771)
  feat(native-filters): add search all filter options (apache#14710)
  fix: extra query in Dashboard when native filter enabled (apache#14770)
  chore: Improves the native filters UI/UX - iteration 2 (apache#14753)
  fix(native filters): Fix explore state (apache#14779)
  fix(explore): DndColumnSelect not handling controls with "multi: false" (apache#14737)
  feat: Create BigQuery Parameters for DatabaseModal (apache#14721)
  feat: enable user impersonation in GSheets (apache#14767)
  fix: add DB should not say it's Postgres (apache#14766)
  Revert "fix(dashboard): multiple query trigger when native filter enabled (apache#14734)" (apache#14762)
  feat: save database with new dynamic form (apache#14583)
  fix: save non-parameter DBs (apache#14759)
  chore: Removes ColorSchemeControl.less (apache#14199)
  fix(explore): Icons width (apache#14717)
  ...
@junlincc junlincc added the dashboard:native-filters Related to the native filters of the Dashboard label May 28, 2021
cccs-RyanS pushed a commit to CybercentreCanada/superset that referenced this pull request Dec 17, 2021
* feat(native-filters): add search all filter options

* add tests

* fix default value

* implement ILIKE operator

* rebump packages

* fix test

* address comments

* fix state changes coming from application

* fix debouncer
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 29, 2021
* feat(native-filters): add search all filter options

* add tests

* fix default value

* implement ILIKE operator

* rebump packages

* fix test

* address comments

* fix state changes coming from application

* fix debouncer
cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024
* feat(native-filters): add search all filter options

* add tests

* fix default value

* implement ILIKE operator

* rebump packages

* fix test

* address comments

* fix state changes coming from application

* fix debouncer
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.3.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels dashboard:native-filters Related to the native filters of the Dashboard preset-io size/L 🚢 1.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants