Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Uptime] Set ML anomaly look-back to 2w (from 24h) / Add spinner #65055

Merged
merged 3 commits into from
May 5, 2020

Conversation

shahzad31
Copy link
Contributor

Summary

Fixes: #64657

Increase anomaly job time analyzing period to 2 weeks.
Also added loading state while job is being created or loaded.

@shahzad31 shahzad31 requested a review from andrewvc May 4, 2020 10:37
@shahzad31 shahzad31 self-assigned this May 4, 2020
@shahzad31 shahzad31 added release_note:fix Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v7.8.0 v8.0.0 and removed release_note:fix labels May 4, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/uptime (Team:uptime)

@shahzad31 shahzad31 added the release_note:skip Skip the PR/issue when compiling release notes label May 4, 2020
@andrewvc andrewvc changed the title [Uptime] Increase anomaly job time analyzing period to 2 weeks [Uptime] Set ML anomaly look-back to 2w (from 24h) / Add spinner May 5, 2020
Copy link
Contributor

@andrewvc andrewvc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks great. Is there a place in ML where I can verify that the right time value is set to 2 weeks?

@shahzad31
Copy link
Contributor Author

@andrewvc yes you can, see the datafeed starting time in below screenshot

image

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / kibana-oss-agent / Firefox UI Functional Tests.test/functional/apps/visualize/input_control_vis/chained_controls·js.visualize app input controls chained controls should disable child control when parent control is not set

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:11:38]         └-: visualize app
[00:11:38]           └-> "before all" hook
[00:11:38]           └-> "before all" hook
[00:11:38]             │ debg Starting visualize before method
[00:11:38]             │ info [logstash_functional] Loading "mappings.json"
[00:11:38]             │ info [logstash_functional] Loading "data.json.gz"
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [logstash-2015.09.22] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:11:38]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.22][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.22][0]]"
[00:11:38]             │ info [logstash_functional] Created index "logstash-2015.09.22"
[00:11:38]             │ debg [logstash_functional] "logstash-2015.09.22" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [logstash-2015.09.20] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:11:38]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.20][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.20][0]]"
[00:11:38]             │ info [logstash_functional] Created index "logstash-2015.09.20"
[00:11:38]             │ debg [logstash_functional] "logstash-2015.09.20" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [logstash-2015.09.21] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:11:38]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.21][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.21][0]]"
[00:11:38]             │ info [logstash_functional] Created index "logstash-2015.09.21"
[00:11:38]             │ debg [logstash_functional] "logstash-2015.09.21" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:11:48]             │ info progress: 13546
[00:11:48]             │ info [logstash_functional] Indexed 4633 docs into "logstash-2015.09.22"
[00:11:48]             │ info [logstash_functional] Indexed 4757 docs into "logstash-2015.09.20"
[00:11:48]             │ info [logstash_functional] Indexed 4614 docs into "logstash-2015.09.21"
[00:11:49]             │ info [long_window_logstash] Loading "mappings.json"
[00:11:49]             │ info [long_window_logstash] Loading "data.json.gz"
[00:11:49]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:11:49]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [long-window-logstash-0] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:11:49]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[long-window-logstash-0][0]]])." previous.health="YELLOW" reason="shards started [[long-window-logstash-0][0]]"
[00:11:49]             │ info [long_window_logstash] Created index "long-window-logstash-0"
[00:11:49]             │ debg [long_window_logstash] "long-window-logstash-0" settings {"index":{"analysis":{"analyzer":{"makelogs_url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:11:59]             │ info progress: 11740
[00:12:00]             │ info [long_window_logstash] Indexed 14005 docs into "long-window-logstash-0"
[00:12:01]             │ info [visualize] Loading "mappings.json"
[00:12:01]             │ info [visualize] Loading "data.json"
[00:12:01]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/7ytHDntrSl6UO8iNC4BMgw] deleting index
[00:12:01]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_1/It6fO_jbTn2VcabpFBTiDQ] deleting index
[00:12:01]             │ info [visualize] Deleted existing index [".kibana_2",".kibana_1"]
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:12:01]             │ info [visualize] Created index ".kibana"
[00:12:01]             │ debg [visualize] ".kibana" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:12:01]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana/W7_rPeMHSpi5iZwOvFC1Bg] update_mapping [_doc]
[00:12:01]             │ info [visualize] Indexed 12 docs into ".kibana"
[00:12:01]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana/W7_rPeMHSpi5iZwOvFC1Bg] update_mapping [_doc]
[00:12:01]             │ debg Migrating saved objects
[00:12:01]             │ proc [kibana]   log   [10:28:12.286] [info][savedobjects-service] Creating index .kibana_2.
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:12:01]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] updating number_of_replicas to [0] for indices [.kibana_2]
[00:12:01]             │ proc [kibana]   log   [10:28:12.332] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:12:01]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] updating number_of_replicas to [0] for indices [.kibana_1]
[00:12:01]             │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] 9154 finished with response BulkByScrollResponse[took=25ms,timed_out=false,sliceId=null,updated=0,created=12,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:12:01]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana/W7_rPeMHSpi5iZwOvFC1Bg] deleting index
[00:12:01]             │ proc [kibana]   log   [10:28:12.659] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:12:01]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/ZH-BNSAUTM6dTyoW820K0Q] update_mapping [_doc]
[00:12:01]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/ZH-BNSAUTM6dTyoW820K0Q] update_mapping [_doc]
[00:12:01]             │ proc [kibana]   log   [10:28:12.742] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:12:01]             │ proc [kibana]   log   [10:28:12.781] [info][savedobjects-service] Finished in 497ms.
[00:12:01]             │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:12:02]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/ZH-BNSAUTM6dTyoW820K0Q] update_mapping [_doc]
[00:12:03]             │ debg replacing kibana config doc: {"defaultIndex":"logstash-*","format:bytes:defaultPattern":"0,0.[000]b"}
[00:12:03]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/ZH-BNSAUTM6dTyoW820K0Q] update_mapping [_doc]
[00:12:04]           └-: 
[00:12:04]             └-> "before all" hook
[00:12:04]             └-: input controls
[00:12:04]               └-> "before all" hook
[00:12:04]               └-: chained controls
[00:12:04]                 └-> "before all" hook
[00:12:04]                 └-> "before all" hook
[00:12:04]                   │ debg navigating to visualize url: http://localhost:6111/app/kibana#/visualize
[00:12:04]                   │ debg navigate to: http://localhost:6111/app/kibana#/visualize
[00:12:04]                   │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:12:04]                   │ debg ... sleep(700) start
[00:12:05]                   │ debg ... sleep(700) end
[00:12:05]                   │ debg returned from get, calling refresh
[00:12:06]                   │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:12:07]                   │ debg currentUrl = http://localhost:6111/app/kibana#/visualize
[00:12:07]                   │          appUrl = http://localhost:6111/app/kibana#/visualize
[00:12:07]                   │ debg TestSubjects.find(kibanaChrome)
[00:12:07]                   │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:12:09]                   │ debg browser[info] "INFO: 2020-05-05T10:28:20Z
[00:12:09]                   │        Adding connection to http://localhost:6111/elasticsearch
[00:12:09]                   │
[00:12:09]                   │      "
[00:12:12]                   │ debg TestSubjects.find(kibanaChrome)
[00:12:12]                   │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=10000
[00:12:12]                   │ debg ... sleep(501) start
[00:12:12]                   │ debg ... sleep(501) end
[00:12:12]                   │ debg in navigateTo url = http://localhost:6111/app/kibana#/visualize?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:12:12]                   │ debg --- retry.try error: URL changed, waiting for it to settle
[00:12:13]                   │ debg ... sleep(501) start
[00:12:13]                   │ debg ... sleep(501) end
[00:12:13]                   │ debg in navigateTo url = http://localhost:6111/app/kibana#/visualize?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:12:13]                   │ debg TestSubjects.exists(statusPageContainer)
[00:12:13]                   │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:12:16]                   │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:12:16]                   │ debg clickVisualizationByLinkText(chained input control)
[00:12:16]                   │ debg Find.clickByPartialLinkText('chained input control') with timeout=10000
[00:12:16]                   │ debg Find.byPartialLinkText('chained input control')  with timeout=10000
[00:12:17]                   │ debg isGlobalLoadingIndicatorVisible
[00:12:17]                   │ debg TestSubjects.exists(globalLoadingIndicator)
[00:12:17]                   │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:12:17]                   │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:12:17]                   │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:12:17]                 └-> should disable child control when parent control is not set
[00:12:17]                   └-> "before each" hook: global before each
[00:12:17]                   │ debg comboBox.getOptionsList, comboBoxSelector: listControlSelect0
[00:12:17]                   │ debg TestSubjects.find(listControlSelect0)
[00:12:17]                   │ debg Find.findByCssSelector('[data-test-subj="listControlSelect0"]') with timeout=10000
[00:12:18]                   │ debg TestSubjects.click(listControlSelect0)
[00:12:18]                   │ debg Find.clickByCssSelector('[data-test-subj="listControlSelect0"]') with timeout=10000
[00:12:18]                   │ debg Find.findByCssSelector('[data-test-subj="listControlSelect0"]') with timeout=10000
[00:12:20]                   │ debg TestSubjects.exists(~comboBoxOptionsList)
[00:12:20]                   │ debg Find.existsByDisplayedByCssSelector('[data-test-subj~="comboBoxOptionsList"]') with timeout=2500
[00:12:20]                   │ debg TestSubjects.find(~comboBoxOptionsList)
[00:12:20]                   │ debg Find.findByCssSelector('[data-test-subj~="comboBoxOptionsList"]') with timeout=10000
[00:12:20]                   │ warn WebElementWrapper.getVisibleText: The element reference of <div class="euiPanel euiComboBoxOptionsList euiComboBoxOptionsList--bottom"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
[00:12:20]                   │ debg finding element 'By(css selector, [data-test-subj~="comboBoxOptionsList"])' again, 2 attempts left
[00:12:31]                   │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/visualize app  input controls chained controls should disable child control when parent control is not set.png"
[00:12:31]                   │ info Current URL is: http://localhost:6111/app/kibana#/visualize/edit/68305470-87bc-11e9-a991-3b492a7c3e09?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(filters:!(),linked:!f,query:(language:kuery,query:%27%27),uiState:(),vis:(aggs:!(),params:(controls:!((fieldName:geo.src,id:%271559757816862%27,indexPattern:%27logstash-*%27,label:%27%27,options:(dynamicOptions:!t,multiselect:!t,order:desc,size:5,type:terms),parent:%27%27,type:list),(fieldName:clientip,id:%271559757836347%27,indexPattern:%27logstash-*%27,label:%27%27,options:(dynamicOptions:!t,multiselect:!t,order:desc,size:5,type:terms),parent:%271559757816862%27,type:list)),pinFilters:!f,updateFiltersOnChange:!f,useTimeFilter:!f),title:%27chained%20input%20control%27,type:input_control_vis))
[00:12:31]                   │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/visualize app  input controls chained controls should disable child control when parent control is not set.html
[00:12:31]                   └- ✖ fail: "visualize app  input controls chained controls should disable child control when parent control is not set"
[00:12:31]                   │

Stack Trace

{ NoSuchElementError: Unable to locate element: [data-test-subj~="comboBoxOptionsList"]
    at Object.throwDecodedError (node_modules/selenium-webdriver/lib/error.js:550:15)
    at parseHttpResponse (node_modules/selenium-webdriver/lib/http.js:565:13)
    at Executor.execute (node_modules/selenium-webdriver/lib/http.js:491:26)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  name: 'NoSuchElementError',
  remoteStacktrace:
   'WebDriverError@chrome://marionette/content/error.js:175:5\nNoSuchElementError@chrome://marionette/content/error.js:387:5\nelement.find/</<@chrome://marionette/content/element.js:331:16\n' }

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@andrewvc andrewvc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shahzad31 shahzad31 merged commit 62574f3 into elastic:master May 5, 2020
@shahzad31 shahzad31 deleted the ml-job-time branch May 5, 2020 12:33
shahzad31 added a commit to shahzad31/kibana that referenced this pull request May 5, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request May 5, 2020
* master: (133 commits)
  Cleanup Typescript index pattern field editor / Expression functions for bucket agg (elastic#65254)
  Removes legacy infra plugin and moves saved objects registration to NP (elastic#64848)
  Added support for docLinks plugin in Connectors forms and missing save capabilities for modal dialog (elastic#64986)
  [SIEM] Removes prebuilt rules number dependency (elastic#65128)
  [Maps] add categorical palettes with 20 and 30 categories (elastic#64701)
  [CI] Slack alerts - Elasticsearch snapshot failures (elastic#64724)
  [Uptime] Console errors in case index missing (elastic#65115)
  [SIEM][CASE] Dynamic fields mapping based on connector (elastic#64412)
  [test/functional] Tsfy page objects (elastic#64887)
  [Maps] [Telemetry] Track geo_point and geo_shape index patterns separately (elastic#65195)
  [Maps] Add global fit to data (elastic#64702)
  Visualize: Reload on ui state change and fix ui state for tsvb (elastic#63699)
  [SIEM] [Cases] External service selection per case (elastic#64775)
  [Uptime] Set ML anomaly look-back to 2w (from 24h) / Add spinner (elastic#65055)
  [Metrics UI] Remove APM Hard Dependency (elastic#64952)
  [Ingest] Datastream list: add icons and dashboard links (elastic#65048)
  disable plugins. they could access ES via SO repository (elastic#65242)
  Feature fleet enrollment instructions (elastic#65176)
  [SIEM] Adds 'Configure connector' Cypress test (elastic#64807)
  [TSVB] Fix std deviation band mode (elastic#64413)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v7.8.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Uptime] Increase ML Window from 24h to 2 Weeks
4 participants