From bcdd769e84c149c5690323e95b43fad25db6d5c2 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 10 Jul 2019 10:11:04 +0200 Subject: [PATCH 01/11] Add 7.3 release highlights. Currently: - votinng-only nodes - replication of aliases by CCR - memory leak with DLS --- .../release-notes/highlights-7.3.0.asciidoc | 45 +++++++++++++++++++ .../release-notes/highlights.asciidoc | 4 +- 2 files changed, 48 insertions(+), 1 deletion(-) create mode 100644 docs/reference/release-notes/highlights-7.3.0.asciidoc diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc new file mode 100644 index 0000000000000..bd10b272b852a --- /dev/null +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -0,0 +1,45 @@ +[[release-highlights-7.3.0]] +== 7.3.0 release highlights +++++ +7.3.0 +++++ + +coming[7.3.0] + +//NOTE: The notable-highlights tagged regions are re-used in the +//Installation and Upgrade Guide + +// tag::notable-highlights[] +[discrete] +==== Voting-only master nodes + +A new <> role has been introduced that +allows nodes to participate in elections even though they are not eligible. +The benefit is that these nodes still help with high availability while +requiring less CPU and heap than master nodes. + +IMPORTANT: The `node.voting-only` role is only available with the default +distribution of Elasticsearch. + +// end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== Aliases are replicated via {ccr} + +Read aliases are now replicated via <>. Note that write +aliases are still not replicated since they only make since for indices that +are being written to while follower indices do not receive direct writes. + +// end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== Memory leaks when using templates in document-level security + +{xpack-ref}/document-level-security.html[Document-level security] was using an +unbounded cache for the set of visible documents. This could lead to memory +leaks when using templated queries as a role query. The cache has been fixed to +evict based on memorory usage and has a limit of 50MB. + +// end::notable-highlights[] diff --git a/docs/reference/release-notes/highlights.asciidoc b/docs/reference/release-notes/highlights.asciidoc index de4ed91d862a0..1b48c8118c61a 100644 --- a/docs/reference/release-notes/highlights.asciidoc +++ b/docs/reference/release-notes/highlights.asciidoc @@ -6,12 +6,14 @@ This section summarizes the most important changes in each release. For the full list, see <> and <>. +* <> * <> * <> * <> -- +include::highlights-7.3.0.asciidoc[] include::highlights-7.2.0.asciidoc[] include::highlights-7.1.0.asciidoc[] -include::highlights-7.0.0.asciidoc[] \ No newline at end of file +include::highlights-7.0.0.asciidoc[] From c867b668eb27b302c9a948642460747d8aff48fd Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 10 Jul 2019 10:15:57 +0200 Subject: [PATCH 02/11] iter --- docs/reference/release-notes/highlights-7.3.0.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index bd10b272b852a..5eba6393374d8 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -10,7 +10,7 @@ coming[7.3.0] //Installation and Upgrade Guide // tag::notable-highlights[] -[discrete] +[float] ==== Voting-only master nodes A new <> role has been introduced that From 919971dc707d907ef79b5d7a3dcda3514436b10f Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 10 Jul 2019 10:16:51 +0200 Subject: [PATCH 03/11] iter --- docs/reference/release-notes/highlights-7.3.0.asciidoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index 5eba6393374d8..4c79239580074 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -35,11 +35,11 @@ are being written to while follower indices do not receive direct writes. // tag::notable-highlights[] [float] -==== Memory leaks when using templates in document-level security +==== Memory leak when using templates in document-level security {xpack-ref}/document-level-security.html[Document-level security] was using an -unbounded cache for the set of visible documents. This could lead to memory -leaks when using templated queries as a role query. The cache has been fixed to +unbounded cache for the set of visible documents. This could lead to a memory +leaks when using a templated query as a role query. The cache has been fixed to evict based on memorory usage and has a limit of 50MB. // end::notable-highlights[] From 4e4048c063edf6b90b06db1fc832481f1d5ba566 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 10 Jul 2019 13:51:33 +0200 Subject: [PATCH 04/11] iter --- docs/reference/release-notes/highlights-7.3.0.asciidoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index 4c79239580074..c091898229bde 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -28,18 +28,18 @@ distribution of Elasticsearch. ==== Aliases are replicated via {ccr} Read aliases are now replicated via <>. Note that write -aliases are still not replicated since they only make since for indices that +aliases are still not replicated since they only make sense for indices that are being written to while follower indices do not receive direct writes. // end::notable-highlights[] // tag::notable-highlights[] [float] -==== Memory leak when using templates in document-level security +==== Fixed memory leak when using templates in document-level seciurity {xpack-ref}/document-level-security.html[Document-level security] was using an unbounded cache for the set of visible documents. This could lead to a memory -leaks when using a templated query as a role query. The cache has been fixed to -evict based on memorory usage and has a limit of 50MB. +leak when using a templated query as a role query. The cache has been fixed to +evict based on memory usage and has a limit of 50MB. // end::notable-highlights[] From 386e31020ab193cead5a8d0c2d52d77d6734c5e7 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 10 Jul 2019 14:20:41 +0200 Subject: [PATCH 05/11] more highlights --- .../release-notes/highlights-7.3.0.asciidoc | 41 ++++++++++++++++++- 1 file changed, 39 insertions(+), 2 deletions(-) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index c091898229bde..ded99daea09ab 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -18,8 +18,36 @@ allows nodes to participate in elections even though they are not eligible. The benefit is that these nodes still help with high availability while requiring less CPU and heap than master nodes. -IMPORTANT: The `node.voting-only` role is only available with the default -distribution of Elasticsearch. +NOTE: The `node.voting-only` role is only available with the default +distribution of {es}. + +// end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== Reloading of search-time synonyms + +A new <> allows to reload the +definition of search-time analyzers and their associated resources. A common +use-case for this API is the reloading of search-time synonyms. In earlier +versions of Elasticsearch, users could force synonyms to be reloaded by closing +the index and then opening it again. With this new API, synonyms can be updated +without closing the index. + +NOTE: The Analyzer reload API is only available with the default distribution +of {es}. + +// end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== Rare terms aggregation + +A new +<> +allows to find the least frequent values in a field. It is intended to replace +the `"order" : { "_count" : "asc" }` option of the +<>. // end::notable-highlights[] @@ -33,6 +61,15 @@ are being written to while follower indices do not receive direct writes. // end::notable-highlights[] +// tag::notable-highlights[] +[float] +==== SQL supports frozen indices + +{es-sql} now supports querying <> via the new +<> keyword. + +// end::notable-highlights[] + // tag::notable-highlights[] [float] ==== Fixed memory leak when using templates in document-level seciurity From 12e25e17b7c8bd56e27a602f691922a5139f6bc9 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 10 Jul 2019 15:37:13 +0200 Subject: [PATCH 06/11] Add note about memory usage of global ords. --- .../release-notes/highlights-7.3.0.asciidoc | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index ded99daea09ab..a0263a176586b 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -72,7 +72,7 @@ are being written to while follower indices do not receive direct writes. // tag::notable-highlights[] [float] -==== Fixed memory leak when using templates in document-level seciurity +==== Fixed memory leak when using templates in document-level security {xpack-ref}/document-level-security.html[Document-level security] was using an unbounded cache for the set of visible documents. This could lead to a memory @@ -80,3 +80,17 @@ leak when using a templated query as a role query. The cache has been fixed to evict based on memory usage and has a limit of 50MB. // end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== More memory-efficient aggregations on `keyword` fields + +<> generally +need to build +<> +in order to run. Unfortunately this operation became more memory-intensive in +6.0 due to the move to doc-value iterators in order to improve handling of +sparse fields. Memory pressure of global ordinals now goes back to a more +similar level as what you could have on pre-6.0 releases. + +// end::notable-highlights[] From 4315a79525ca5f475e89ca3e1f4d00e9e1b7f958 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 10 Jul 2019 16:47:38 +0200 Subject: [PATCH 07/11] Add intervals prefix/wildcard, flattened and vector functions. --- .../release-notes/highlights-7.3.0.asciidoc | 35 +++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index a0263a176586b..c033a660ff16e 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -39,6 +39,41 @@ of {es}. // end::notable-highlights[] +// tag::notable-highlights[] +[float] +==== New `flattened` field type + +A new <> field type has been added, which can index +arbitrary `json` objects into a single field. This helps avoid hitting issues +due to many fields in mappings, at the cost of more limited search +functionality. + +NOTE: The <> field type is only available with the +default distribution of {es}. + +// end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== Functions on vector fields + +Painless now support computing the <> and +the <> of a query vector and either values of a +<> or <> field. + +NOTE: These functions are only available with the default distribution of {es}. + +// end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== Prefix and wildcard support for intervals + +<> now support querying by +<> or <>. + +// end::notable-highlights[] + // tag::notable-highlights[] [float] ==== Rare terms aggregation From c34f14e155a65caa6a5d28bd43c176d2728a48f3 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Thu, 11 Jul 2019 09:38:42 +0200 Subject: [PATCH 08/11] Update docs/reference/release-notes/highlights-7.3.0.asciidoc Co-Authored-By: Tim Vernum --- docs/reference/release-notes/highlights-7.3.0.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index c033a660ff16e..2e004576c601f 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -14,7 +14,7 @@ coming[7.3.0] ==== Voting-only master nodes A new <> role has been introduced that -allows nodes to participate in elections even though they are not eligible. +allows nodes to participate in elections even though they are not eligible to become the master. The benefit is that these nodes still help with high availability while requiring less CPU and heap than master nodes. From ab4154b61b02e0eab9c7169cda2e16e2e8d770c9 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Fri, 19 Jul 2019 15:43:36 +0200 Subject: [PATCH 09/11] Add data frame pivot transforms. --- .../reference/release-notes/highlights-7.3.0.asciidoc | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index 2e004576c601f..c7227eccc819a 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -129,3 +129,14 @@ sparse fields. Memory pressure of global ordinals now goes back to a more similar level as what you could have on pre-6.0 releases. // end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== Data frame pivot transforms to create entity-centric indexes + +<>, released in 7.2, allow to transform an +existing index to a secondary, summarized index. 7.3 now introduces Data frame +pivot transforms in order to create entity-centric indexes that can summarize +the behavior of an entity.  + +// end::notable-highlights[] From 02db0e577e8710eb61703fb7582bf4a53674baed Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Fri, 19 Jul 2019 15:44:15 +0200 Subject: [PATCH 10/11] add license --- docs/reference/release-notes/highlights-7.3.0.asciidoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index c7227eccc819a..94725cfd7ef58 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -139,4 +139,6 @@ existing index to a secondary, summarized index. 7.3 now introduces Data frame pivot transforms in order to create entity-centric indexes that can summarize the behavior of an entity.  +NOTE: Data frames are only available with the default distribution of {es}. + // end::notable-highlights[] From 6c05c54e0112248b28811c1fbb21eba4fe538977 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 24 Jul 2019 09:52:28 +0200 Subject: [PATCH 11/11] Outlier detection. --- .../release-notes/highlights-7.3.0.asciidoc | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/reference/release-notes/highlights-7.3.0.asciidoc b/docs/reference/release-notes/highlights-7.3.0.asciidoc index 94725cfd7ef58..33002224a21d5 100644 --- a/docs/reference/release-notes/highlights-7.3.0.asciidoc +++ b/docs/reference/release-notes/highlights-7.3.0.asciidoc @@ -142,3 +142,18 @@ the behavior of an entity.  NOTE: Data frames are only available with the default distribution of {es}. // end::notable-highlights[] + +// tag::notable-highlights[] +[float] +==== Outlier detection + +{stack-ov}/security-privileges.html[Outlier detection] utilizes Elastic data +frame indexes to evaluate source indexes across multiple dimensions and identify +clusters of data based on the assigned values and which values are different +from those of the clustered data point. An outlier score can be used to indicate +how different an entity is from other entities in the index based on the +dimensions that you supply. + +NOTE: Outlier detection requires a platinum license. + +// end::notable-highlights[]