[ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. #81123

walterra · 2020-10-20T10:13:37Z

Summary

Fixes #81150
Part of #77046

Fixes a race condition where searches for data grid results with different parameters would return in different order with the wrong results on display. Fix uses a pattern to cancel useEffect callback for getIndexData().
Fixes identifying pre 7.10 feature influence format for outlier detection and will display a callout on the results page with information for a workaround.
To fix identifying the legacy format, some cleanup of other code relating to the old format had to be done. The ml results object field is no longer treated as a "special" field for outlier detection and is treated and retrieved in the same way as other fields.
Adds an error callout if no Kibana index pattern is available for source/dest index.

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
This was checked for breaking API changes and was labeled appropriately

…ier influence code.

elasticmachine · 2020-10-20T10:13:39Z

Pinging @elastic/ml-ui (:ml)

...analytics/pages/analytics_exploration/components/outlier_exploration/outlier_exploration.tsx

peteharverson · 2020-10-20T11:00:49Z

x-pack/plugins/ml/public/application/data_frame_analytics/common/get_index_data.ts

-
-      setTableItems(docs);
-      setStatus(INDEX_STATUS.LOADED);
+      if (!options.didCancel) {


For regression and classification jobs, the results grid is failing to display for me:

peteharverson

Tested latest edits and LGTM

peteharverson · 2020-10-20T13:15:52Z

If the index pattern does not exist for the results index, the callout appears even if the feature influence is in the correct format:

darnautov · 2020-10-20T13:26:37Z

.../pages/analytics_exploration/components/exploration_results_table/use_exploration_results.ts

+    const options = { didCancel: false };
+    getIndexData(jobConfig, dataGrid, searchQuery, options);
+    return () => {
+      options.didCancel = true;


shall we create a custom hook, let's call it useSafeEffect for instance, that wraps the callback with didCancel check under the hood?

yep that's something we could do in a follow up for 7.11 - for this 7.10 fix I wanted to touch as little code as possible that's why I went for duplicate code and no new utils. i'd also like to refactor getIndexData a bit so we don't have to use the mutable object but just a plain variable.

peteharverson

Tested latest commit and LGTM

walterra · 2020-10-20T21:39:47Z

@alvarezmelissa87 Added an error callout when both index patterns for source/dest index are missing, ready for review :)

alvarezmelissa87

Left a comment regarding classification/regression results page when there is no source/destination index pattern.

alvarezmelissa87 · 2020-10-20T23:21:04Z

When I navigate to results view for Regression or Classification jobs that don't have a source and destination index I now get a blank page instead of an error callout with a useful message.

alvarezmelissa87

Tested latest edits and LGTM ⚡

… influence legacy format. (elastic#81123) - Fixes a race condition where searches for data grid results with different parameters would return in different order with the wrong results on display. Fix uses a pattern to cancel useEffect callback for getIndexData(). - Fixes identifying pre 7.10 feature influence format for outlier detection and will display a callout on the results page with information for a workaround. - To fix identifying the legacy format, some cleanup of other code relating to the old format had to be done. The ml results object field is no longer treated as a "special" field for outlier detection and is treated and retrieved in the same way as other fields. - Adds an error callout if no Kibana index pattern is available for source/dest index.

… influence legacy format. (#81123) (#81275) - Fixes a race condition where searches for data grid results with different parameters would return in different order with the wrong results on display. Fix uses a pattern to cancel useEffect callback for getIndexData(). - Fixes identifying pre 7.10 feature influence format for outlier detection and will display a callout on the results page with information for a workaround. - To fix identifying the legacy format, some cleanup of other code relating to the old format had to be done. The ml results object field is no longer treated as a "special" field for outlier detection and is treated and retrieved in the same way as other fields. - Adds an error callout if no Kibana index pattern is available for source/dest index.

… influence legacy format. (#81123) (#81276) - Fixes a race condition where searches for data grid results with different parameters would return in different order with the wrong results on display. Fix uses a pattern to cancel useEffect callback for getIndexData(). - Fixes identifying pre 7.10 feature influence format for outlier detection and will display a callout on the results page with information for a workaround. - To fix identifying the legacy format, some cleanup of other code relating to the old format had to be done. The ml results object field is no longer treated as a "special" field for outlier detection and is treated and retrieved in the same way as other fields. - Adds an error callout if no Kibana index pattern is available for source/dest index.

…arm-phase-to-formlib * 'master' of github.com:elastic/kibana: (55 commits) [UX] Fix map color variance and apply proper filter for extended stats (elastic#81106) [User Experience] Use EuiSelect for percentiles instead of SuperSelect (elastic#81082) [DOCS] Add link for monitoring ssl settings (elastic#81057) [test] Await loading indicator in monitoring test (elastic#81279) [ILM] Minor copy and link additions to cloud CTA for cold phase (elastic#80512) [Mappings editor] Add scaled_float and date_range comp integration tests (elastic#81287) [Discover] Deangularize context.app (elastic#80851) [O11y Overview] Add code to display/hide UX section when appropriate (elastic#80873) [Discover] Extend DiscoverNoResults component to show different message on error (elastic#79671) Fix tagcloud word overlapping (elastic#81161) [Security Solution] Fixes flaky test rules (elastic#81040) Changed the code to avoid tech debt with hacky solutions after receiving comments on EUI issue reported about this problem. (elastic#81183) [Security Solution][All] Replace old markdown renderer with the new one (elastic#80301) Add namespaced version of the API call (elastic#81278) [ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. (elastic#81123) [Fleet] Fix POLICY_CHANGE action creation for new policy (elastic#81236) [Security Solution][Endpoint][Admin] Malware user notification checkbox (elastic#78084) [SecuritySolution][Unit Tests] - fix flakey unit test (elastic#81239) skip flaky suite (elastic#81264) [Maps] fix top-level Map page is called 'Kibana' (elastic#81238) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/hot_phase/hot_phase.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/shared/forcemerge_field.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/warm_phase.tsx

kibanamachine · 2020-10-22T11:50:29Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: dbbc3d0

Metrics [docs]

async chunks size

id	before	after	diff
`ml`	7.3MB	7.3MB	+6.3KB

History

💚 Build #82962 succeeded dbbc3d0
💚 Build #82934 succeeded dea4599
💚 Build #82765 succeeded 625dc09
💚 Build #82732 succeeded 6a6255c
💔 Build #82710 failed 0785fb7

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

walterra added 2 commits October 20, 2020 12:05

[ML] Fix race condition when loading index data. Clean up legacy outl…

de33e97

…ier influence code.

[ML] Add callout for legacy feature influence data.

0785fb7

walterra added bug Fixes for quality problems that affect the customer experience regression :ml v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Feature:Data Frame Analytics ML data frame analytics features v7.10.0 v7.11.0 labels Oct 20, 2020

walterra requested review from alvarezmelissa87 and peteharverson October 20, 2020 10:13

walterra requested a review from a team as a code owner October 20, 2020 10:13

walterra self-assigned this Oct 20, 2020

peteharverson reviewed Oct 20, 2020

View reviewed changes

...analytics/pages/analytics_exploration/components/outlier_exploration/outlier_exploration.tsx Outdated Show resolved Hide resolved

peteharverson reviewed Oct 20, 2020

View reviewed changes

walterra added 2 commits October 20, 2020 13:54

[ML] Fix cancelling regression/classification results.

2768a72

[ML] Tweak legacy callout text.

6a6255c

peteharverson approved these changes Oct 20, 2020

View reviewed changes

peteharverson self-requested a review October 20, 2020 13:16

darnautov reviewed Oct 20, 2020

View reviewed changes

[ML] Hide legacy callout if index pattern is missing.

625dc09

peteharverson approved these changes Oct 20, 2020

View reviewed changes

[ML] Show callout when both index patterns are missing.

dea4599

alvarezmelissa87 requested changes Oct 20, 2020

View reviewed changes

[ML] Fix callout for regression/classification.

dbbc3d0

alvarezmelissa87 approved these changes Oct 21, 2020

View reviewed changes

walterra merged commit 4ba2fdb into elastic:master Oct 21, 2020

walterra deleted the ml-analytics-outlier-legacy-format branch October 21, 2020 05:57

This was referenced Oct 21, 2020

[7.x] [ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. (#81123) #81275

Merged

[7.10] [ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. (#81123) #81276

Merged

This was referenced Oct 22, 2020

[ML] Data Frame Analytics results UI - no warning if no index pattern exists #81150

Closed

[ML] Dataframe Analytics: Update feature_influence format #77046

Closed

walterra mentioned this pull request Oct 29, 2020

[ML] Data Frame Analytics: Fix feature importance cell value and decision path chart #82011

Merged

2 tasks

walterra mentioned this pull request Jan 19, 2021

[ML] Improving model snapshot revert UI experience #88588

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. #81123

[ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. #81123

walterra commented Oct 20, 2020 •

edited

Loading

elasticmachine commented Oct 20, 2020

peteharverson Oct 20, 2020

peteharverson left a comment

peteharverson commented Oct 20, 2020

darnautov Oct 20, 2020

walterra Oct 20, 2020

peteharverson left a comment

walterra commented Oct 20, 2020

alvarezmelissa87 left a comment

alvarezmelissa87 commented Oct 20, 2020

alvarezmelissa87 left a comment

kibanamachine commented Oct 22, 2020

[ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. #81123

[ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. #81123

Conversation

walterra commented Oct 20, 2020 • edited Loading

Summary

Checklist

elasticmachine commented Oct 20, 2020

peteharverson Oct 20, 2020

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

peteharverson commented Oct 20, 2020

darnautov Oct 20, 2020

Choose a reason for hiding this comment

walterra Oct 20, 2020

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

walterra commented Oct 20, 2020

alvarezmelissa87 left a comment

Choose a reason for hiding this comment

alvarezmelissa87 commented Oct 20, 2020

alvarezmelissa87 left a comment

Choose a reason for hiding this comment

kibanamachine commented Oct 22, 2020

💚 Build Succeeded

Metrics [docs]

async chunks size

History

walterra commented Oct 20, 2020 •

edited

Loading