Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metrics UI] Anomaly Detection setup flow for Metrics #76787

Merged
merged 33 commits into from
Sep 23, 2020

Conversation

phillipb
Copy link
Contributor

@phillipb phillipb commented Sep 4, 2020

Summary

Adds the ability to set up hosts and k8s anomaly detection jobs on the inventory screen, and adds endpoints and hooks for fetching anomalies associated with those jobs. The PR sets up an infra_ml directory that could become a shared directory with logs code in the future. There is a good amount of copying from logs taking place, and it would be good at some future date to refactor/dedupe some things. Fixes: #74808, #74807

Screen Shot 2020-09-21 at 9 11 46 AM

Known issues

Checklist

Delete any items that are not applicable to this PR.

@phillipb phillipb added release_note:enhancement Feature:Metrics UI Metrics UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v7.10.0 labels Sep 4, 2020
@phillipb phillipb requested review from a team as code owners September 4, 2020 15:57
@elasticmachine
Copy link
Contributor

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

@phillipb
Copy link
Contributor Author

phillipb commented Sep 8, 2020

@elasticmachine merge upstream

@phillipb phillipb removed the request for review from a team September 21, 2020 16:13
phillipb and others added 2 commits September 21, 2020 17:51
- Remove server/lib/snapshot
- Replace backend for /api/infra/snapshot with data from Metrics API
- Fixing tests with updates to the snapshot node

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@phillipb
Copy link
Contributor Author

@elasticmachine merge upstream

@simianhacker simianhacker self-assigned this Sep 22, 2020
Copy link
Member

@simianhacker simianhacker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should replace all the npStart.http.fetch requests with useHTTPRequest hook (under public/hooks). There seems to be a lot of duplicate functionality and it will make it easier update later.

@phillipb
Copy link
Contributor Author

@simianhacker switching to useHTTPRequest isn't really clean with the way the code is structured. useHTTPRequest is a hook with state, but these calls to npStart.http.fetch are expected to flow more procedurally. They chain together which makes it really hard to do via effects. For example, https://github.com/phillipb/kibana/blob/81bc98860447f6d6871108a751d50204c2fdc259/x-pack/plugins/infra/public/containers/ml/infra_ml_cleanup.tsx#L10. cleanUpAndSetUpModule which calls callStopDatafeeds and then calls deleteJobs and then calls, setUpModule and then calls getJobSummary. Cleaner and easier to write that without hooks.

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

@kbn/optimizer bundle module count

id value diff baseline
infra 1121 +25 1096

async chunks size

id value diff baseline
infra 4.0MB ⚠️ +228.4KB 3.8MB

distributable file count

id value diff baseline
default 45968 +26 45942

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Member

@simianhacker simianhacker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested it with a RO user and Admin and everything works as expected. Good Job!

███▄▄▄▄      ▄████████  ▄█   ▄█          ▄████████ ████████▄        ▄█      ███     
███▀▀▀██▄   ███    ███ ███  ███         ███    ███ ███   ▀███      ███  ▀█████████▄ 
███   ███   ███    ███ ███▌ ███         ███    █▀  ███    ███      ███▌    ▀███▀▀██ 
███   ███   ███    ███ ███▌ ███        ▄███▄▄▄     ███    ███      ███▌     ███   ▀ 
███   ███ ▀███████████ ███▌ ███       ▀▀███▀▀▀     ███    ███      ███▌     ███     
███   ███   ███    ███ ███  ███         ███    █▄  ███    ███      ███      ███     
███   ███   ███    ███ ███  ███▌    ▄   ███    ███ ███   ▄███      ███      ███     
 ▀█   █▀    ███    █▀  █▀   █████▄▄██   ██████████ ████████▀       █▀      ▄████▀   
                            ▀                                                       
                ▄█          ▄██████▄      ███       ▄▄▄▄███▄▄▄▄                     
               ███         ███    ███ ▀█████████▄ ▄██▀▀▀███▀▀▀██▄                   
               ███         ███    █▀     ▀███▀▀██ ███   ███   ███                   
               ███        ▄███            ███   ▀ ███   ███   ███                   
               ███       ▀▀███ ████▄      ███     ███   ███   ███                   
               ███         ███    ███     ███     ███   ███   ███                   
               ███▌    ▄   ███    ███     ███     ███   ███   ███                   
               █████▄▄██   ████████▀     ▄████▀    ▀█   ███   █▀                    
               ▀                                                                    

@phillipb phillipb merged commit 14921a0 into elastic:master Sep 23, 2020
phillipb added a commit to phillipb/kibana that referenced this pull request Sep 24, 2020
* adds metrics ml integration

* Add ability to create ml jobs from inventory

* Fix i18n stuff

* Fix typecheck

* renames jobs, updates datafeeds

* adds allow_no_indices: true for datafeeds

* Revert "[Metrics UI] Replace Snapshot API with Metrics API (elastic#76253)"

This reverts commit 0ca6472.

* Add ability to fetch anomalies

* Fix typecheck

* Fix typecheck

* Fix i18n

* Fix lint, use the right partition field

* Delete log files

* Fix merge

* Fix merge issues

* Update name of jobs

* Remove CPU job

* [Metrics UI] Replace Snapshot API with Metrics API (elastic#76253)

- Remove server/lib/snapshot
- Replace backend for /api/infra/snapshot with data from Metrics API
- Fixing tests with updates to the snapshot node

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* Add links back to ML for anomalies and manage jobs

* Fix typecheck

* Remove unecessary validation

Co-authored-by: Michael Hirsch <michaelahirsch@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Chris Cowan <chris@chriscowan.us>
phillipb added a commit that referenced this pull request Sep 24, 2020
* adds metrics ml integration

* Add ability to create ml jobs from inventory

* Fix i18n stuff

* Fix typecheck

* renames jobs, updates datafeeds

* adds allow_no_indices: true for datafeeds

* Revert "[Metrics UI] Replace Snapshot API with Metrics API (#76253)"

This reverts commit 0ca6472.

* Add ability to fetch anomalies

* Fix typecheck

* Fix typecheck

* Fix i18n

* Fix lint, use the right partition field

* Delete log files

* Fix merge

* Fix merge issues

* Update name of jobs

* Remove CPU job

* [Metrics UI] Replace Snapshot API with Metrics API (#76253)

- Remove server/lib/snapshot
- Replace backend for /api/infra/snapshot with data from Metrics API
- Fixing tests with updates to the snapshot node

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* Add links back to ML for anomalies and manage jobs

* Fix typecheck

* Remove unecessary validation

Co-authored-by: Michael Hirsch <michaelahirsch@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Chris Cowan <chris@chriscowan.us>

Co-authored-by: Michael Hirsch <michaelahirsch@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Chris Cowan <chris@chriscowan.us>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Metrics UI Metrics UI feature release_note:enhancement Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v7.10.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Metrics UI] Setup screen for ML Job
6 participants