[ILM] Add support for frozen phase in UI #75968

jloleysens · 2020-08-26T10:57:51Z

Summary

Support configuring the frozen phase in both server and UI of ILM. This also updates the index management plugins extension to support querying for "frozen" phase in the index table.

How to test

Create a new ILM policy that consists of Hot and Frozen phases only (to simplify testing). In the UI, make sure that you use all options for frozen phase except for setting required replicas as this will cause the phase to get stuck there if the cluster cannot create a replica of the shards.
Follow this tutorial to set up an index template that will use the ILM policy and create an initial write index (making sure to keep your naming consistent). Make sure to set any timing values to one second (1 second) so that you do not have to wait for phase rollover too long.
Run the following in Dev Tools to make ILM check progress more frequently:

PUT _cluster/settings
{
  "transient": {
    "indices.lifecycle.poll_interval": "5s"  
  }
}

Make sure to set this back to the default of 10m so that ILM doesn't continue doing unnecessary work after testing.

What you should see in index management is new indices being created every 5 seconds as rollover happens and they should have the "Frozen" label applied to them in the index management UI.

Fix elastic/elasticsearch#61345

Screenshots

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios

This is a very faithful duplication of the cold phase. We are also excluding the snapshot action as well as the unfollow action as these are features that require higher than basic license privilege.

elasticmachine · 2020-08-26T10:57:53Z

Pinging @elastic/es-ui (Team:Elasticsearch UI)

jloleysens · 2020-08-27T07:02:03Z

@elasticmachine merge upstream

yuliacech · 2020-08-27T11:23:33Z

Hi @jloleysens ,
I tested the code locally with a data stream and the frozen phase worked for me.
I noticed some issues with validation and typings, I will add comments below.

yuliacech · 2020-08-27T11:27:20Z

...s/index_lifecycle_management/public/application/sections/edit_policy/phases/frozen_phase.tsx

+          <Fragment>
+            {phaseData.phaseEnabled ? (
+              <Fragment>
+                <MinAgeInput<FrozenPhaseInterface>


MinAgeInput's props are currently typed to accept warm, cold and delete phases. I'm not sure why the type check is not picking it up, but it might be good to add frozen phase to props generics.

yuliacech · 2020-08-27T11:28:30Z

...s/index_lifecycle_management/public/application/sections/edit_policy/phases/frozen_phase.tsx

+                />
+                <EuiSpacer />
+
+                <NodeAllocation<FrozenPhaseInterface>


Similar here, the props generics don't include frozen phase, but type check is not showing it.

...plugins/index_lifecycle_management/public/application/services/policies/policy_validation.ts

jloleysens · 2020-08-27T11:52:36Z

@elasticmachine merge upstream

- also fixed TS error introduced after TS v4 upgrade (delete)

jloleysens · 2020-08-27T12:43:10Z

@yuliacech Thanks for the review and great catch on frozen phase validation! I've addressed your PR feedback, would you mind taking another look?

I did a slight restructure to the phase types. Because there is so much overlap I thought it would be easiest to use inheritance for the phase interfaces. So now when we create a new "shared" form UI component when can just extend the relevant phases and structurally the phases that extend that type should fit into the component type requirements. Let me know what you think!

yuliacech

Hi @jloleysens ,
Great addition to policy types with common phase interfaces! Tested locally and all looks good to me.
I just found a copy-paste error from my commit, so I wont block you on it. But if you want to fix it with this PR, here is the info: both validateColdPhase and validateFrozenPhase when checking phase.selectedMinimumAge actually setting phaseErrors.phaseIndexPriority.

jloleysens · 2020-08-27T17:35:00Z

Thanks Yulia! I will include the fix and merge once CI is green.

cjcenizal · 2020-08-27T18:21:16Z

x-pack/plugins/index_lifecycle_management/public/application/services/policies/frozen_phase.ts

+    }
+  }
+
+  if (


I would love to see these kinds of rules extracted into functions that have descriptive names, a big block comment explaining the intention (with links to relevant docs), and then covered with unit tests to define the desired behavior. I feel like this would make the code easier to scan, and it'd also be easier to dig into them to understand the why behind these rules. Anybody else feel similarly?

That would be indeed very helpful 👍

I agree, I think we should also move the files to directory called lib in that process that lives in application. For now, the logic is a carbon-copy of that for cold phase, so at least we know it has been working. I will open an issue for this!

jloleysens · 2020-08-31T07:12:55Z

@elasticmachine merge upstream

jloleysens · 2020-08-31T07:32:38Z

@elasticmachine merge upstream

kibanamachine · 2020-08-31T09:15:55Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: a087b04

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
indexLifecycleManagement	159	+2	157

async chunks size

id	value	diff	baseline
indexLifecycleManagement	276.0KB	+16.1KB	259.9KB

page load bundle size

id	value	diff	baseline
indexLifecycleManagement	230.2KB	+430.0B	229.8KB

History

💚 Build #70973 succeeded dc2b0d1
💚 Build #70824 succeeded c366c5e
💔 Build #70817 failed 067d25a
💚 Build #70777 succeeded 5149dd2
💔 Build #70477 failed 9ff7eaa

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* ILM public side changes to support frozen phase This is a very faithful duplication of the cold phase. We are also excluding the snapshot action as well as the unfollow action as these are features that require higher than basic license privilege. * added "frozen" to the server side schema * add frozen phases component * fix i18n and update jest tests * Slight restructuring to phase types and fix copy paste issues. - also fixed TS error introduced after TS v4 upgrade (delete) * fix hot phase type and remove type constraint from error messages * update validation logic Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

…s-for-710 * 'master' of github.com:elastic/kibana: (43 commits) [APM] Chart units don't update when toggling the chart legends (elastic#74931) [ILM] Add support for frozen phase in UI (elastic#75968) Hides advanced json for count metric (elastic#74636) add client-side feature usage API (elastic#75486) [Maps] add drilldown support map embeddable (elastic#75598) [Enterprise Search] Request handler refactors/enhancements + update existing routes to use shared handler (elastic#76106) [Resolver] model `location.search` in redux (elastic#76140) [APM] Prevent imports of public in server code (elastic#75979) fix eslint issue skip flaky suite (elastic#76223) [APM] Transaction duration anomaly alerting integration (elastic#75719) [Transforms] Avoid using "Are you sure" (elastic#75932) [Security Solution][Exceptions] - Fix bug of alerts not updating after closure from exceptions modal (elastic#76145) [plugin-helpers] improve 3rd party KP plugin support (elastic#75019) [docs/getting-started] link to yarn v1 specifically (elastic#76169) [Security_Solution][Resolver] Resolver loading and error state (elastic#75600) Fixes App Search documentation links (elastic#76133) Fix alerts unable to create / update when the name has trailing whitepace(s) (elastic#76079) [Resolver] Fix useSelector usage (elastic#76129) [Enterprise Search] Migrate util and components from ent-search (elastic#76051) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/data_tier_allocation/node_allocation.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/edit_policy.tsx # x-pack/plugins/index_lifecycle_management/public/application/services/policies/types.ts # x-pack/plugins/index_lifecycle_management/public/application/services/policies/warm_phase.ts

* ILM public side changes to support frozen phase This is a very faithful duplication of the cold phase. We are also excluding the snapshot action as well as the unfollow action as these are features that require higher than basic license privilege. * added "frozen" to the server side schema * add frozen phases component * fix i18n and update jest tests * Slight restructuring to phase types and fix copy paste issues. - also fixed TS error introduced after TS v4 upgrade (delete) * fix hot phase type and remove type constraint from error messages * update validation logic Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

jloleysens added 2 commits August 26, 2020 12:46

ILM public side changes to support frozen phase

469e5bb

This is a very faithful duplication of the cold phase. We are also excluding the snapshot action as well as the unfollow action as these are features that require higher than basic license privilege.

added "frozen" to the server side schema

19afe82

jloleysens added chore Feature:ILM Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more release_note:skip Skip the PR/issue when compiling release notes labels Aug 26, 2020

jloleysens requested a review from cjcenizal August 26, 2020 10:57

jloleysens requested a review from a team as a code owner August 26, 2020 10:57

jloleysens added v7.10.0 v8.0.0 labels Aug 26, 2020

add frozen phases component

9ff7eaa

jloleysens requested review from yuliacech and removed request for cjcenizal August 27, 2020 06:54

fix i18n and update jest tests

e7dcc64

Merge branch 'master' into ilm/add-frozen-phase

5149dd2

yuliacech reviewed Aug 27, 2020

View reviewed changes

...plugins/index_lifecycle_management/public/application/services/policies/policy_validation.ts Outdated Show resolved Hide resolved

yuliacech reviewed Aug 27, 2020

View reviewed changes

...plugins/index_lifecycle_management/public/application/services/policies/policy_validation.ts Show resolved Hide resolved

elasticmachine and others added 3 commits August 27, 2020 05:52

Merge branch 'master' into ilm/add-frozen-phase

067d25a

Slight restructuring to phase types and fix copy paste issues.

0935f6b

- also fixed TS error introduced after TS v4 upgrade (delete)

fix hot phase type and remove type constraint from error messages

c366c5e

yuliacech approved these changes Aug 27, 2020

View reviewed changes

update validation logic

dc2b0d1

jloleysens mentioned this pull request Aug 27, 2020

[ILM] Data tiers for 7.10 #76126

Merged

11 tasks

cjcenizal reviewed Aug 27, 2020

View reviewed changes

Merge branch 'master' into ilm/add-frozen-phase

a087b04

jloleysens merged commit aac57fb into elastic:master Aug 31, 2020

jloleysens deleted the ilm/add-frozen-phase branch August 31, 2020 09:46

This was referenced Aug 31, 2020

[ILM] Refactor phase serialization logic and add tests #76267

Closed

[7.x] [ILM] Add support for frozen phase in UI (#75968) #76268

Merged

jloleysens mentioned this pull request Sep 18, 2020

[ILM] Remove frozen phase #77877

Merged

cjcenizal mentioned this pull request Feb 11, 2021

Add frozen phase to ILM UI #91189

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ILM] Add support for frozen phase in UI #75968

[ILM] Add support for frozen phase in UI #75968

jloleysens commented Aug 26, 2020 •

edited

Loading

elasticmachine commented Aug 26, 2020

jloleysens commented Aug 27, 2020

yuliacech commented Aug 27, 2020

yuliacech Aug 27, 2020

yuliacech Aug 27, 2020

jloleysens commented Aug 27, 2020

jloleysens commented Aug 27, 2020 •

edited

Loading

yuliacech left a comment

jloleysens commented Aug 27, 2020

cjcenizal Aug 27, 2020

sebelga Aug 31, 2020

jloleysens Aug 31, 2020

jloleysens commented Aug 31, 2020

jloleysens commented Aug 31, 2020

kibanamachine commented Aug 31, 2020

+                  }
+                }
+                if (

[ILM] Add support for frozen phase in UI #75968

[ILM] Add support for frozen phase in UI #75968

Conversation

jloleysens commented Aug 26, 2020 • edited Loading

Summary

How to test

Screenshots

Checklist

elasticmachine commented Aug 26, 2020

jloleysens commented Aug 27, 2020

yuliacech commented Aug 27, 2020

yuliacech Aug 27, 2020

Choose a reason for hiding this comment

yuliacech Aug 27, 2020

Choose a reason for hiding this comment

jloleysens commented Aug 27, 2020

jloleysens commented Aug 27, 2020 • edited Loading

yuliacech left a comment

Choose a reason for hiding this comment

jloleysens commented Aug 27, 2020

cjcenizal Aug 27, 2020

Choose a reason for hiding this comment

sebelga Aug 31, 2020

Choose a reason for hiding this comment

jloleysens Aug 31, 2020

Choose a reason for hiding this comment

jloleysens commented Aug 31, 2020

jloleysens commented Aug 31, 2020

kibanamachine commented Aug 31, 2020

💚 Build Succeeded

Build metrics

@kbn/optimizer bundle module count

async chunks size

page load bundle size

History

jloleysens commented Aug 26, 2020 •

edited

Loading

jloleysens commented Aug 27, 2020 •

edited

Loading