Merge branch 'master' into ccr

* master:
  Remove reference to non-existent store type (#32418)
  [TEST] Mute failing FlushIT test
  Fix ordering of bootstrap checks in docs (#32417)
  [TEST] Mute failing InternalEngineTests#testSeqNoAndCheckpoints
  [TEST] Mute failing testConvertLongHexError
  bump lucene version after backport
  Upgrade to Lucene-7.5.0-snapshot-608f0277b0 (#32390)
  [Kerberos] Avoid vagrant update on precommit (#32416)
  TESTS: Move netty leak detection to paranoid level (#32354)
  [DOCS] Fixes formatting of scope object in job resource
  Copy missing segment attributes in getSegmentInfo (#32396)
  AbstractQueryTestCase should run without type less often (#28936)
  INGEST: Fix Deprecation Warning in Script Proc. (#32407)
  Switch x-pack/plugin to new style Requests (#32327)
  Docs: Correcting a typo in tophits (#32359)
  Build: Stop double generating buildSrc pom (#32408)
  TEST: Avoid triggering merges in FlushIT
  Fix missing JavaDoc for @throws in several places in KerberosTicketValidator.
  Switch x-pack full restart to new style Requests (#32294)
  Release requests in cors handler (#32364)
  Painless: Clean Up PainlessClass Variables (#32380)
  Docs: Fix callouts in put license HL REST docs (#32363)
  [ML] Consistent pattern for strict/lenient parser names (#32399)
  Update update-settings.asciidoc (#31378)
  Remove some dead code (#31993)
  Introduce index store plugins (#32375)
  Rank-Eval: Reduce scope of an unchecked supression
  Make sure _forcemerge respects `max_num_segments`. (#32291)
  TESTS: Fix Buf Leaks in HttpReadWriteHandlerTests (#32377)
  Only enforce password hashing check if FIPS enabled (#32383)

buildSrc/build.gradle

@@ -183,4 +183,12 @@ if (project != rootProject) {
     testClass = 'org.elasticsearch.gradle.test.GradleUnitTestCase'
     integTestClass = 'org.elasticsearch.gradle.test.GradleIntegrationTestCase'
   }
+
+  /*
+   * We alread configure publication and we don't need or want this one that
+   * comes from the java-gradle-plugin.
+   */
+  afterEvaluate {
+    generatePomFileForPluginMavenPublication.enabled = false
+  }
 }

buildSrc/version.properties

@@ -1,5 +1,5 @@
 elasticsearch     = 7.0.0-alpha1
-lucene            = 7.5.0-snapshot-b9e064b935
+lucene            = 7.5.0-snapshot-608f0277b0
 
 # optional dependencies
 spatial4j         = 0.7

docs/java-rest/high-level/licensing/put-license.asciidoc

@@ -33,10 +33,9 @@ include-tagged::{doc-tests}/LicensingDocumentationIT.java[put-license-response]
 --------------------------------------------------
 <1> The status of the license
 <2> Make sure that the license is valid.
-<3> Check the acknowledge flag.
-<4> It should be true if license is acknowledge.
-<5> Otherwise we can see the acknowledge messages in `acknowledgeHeader()` and check
-component-specific messages in `acknowledgeMessages()`.
+<3> Check the acknowledge flag. It should be true if license is acknowledged.
+<4> Otherwise we can see the acknowledge messages in `acknowledgeHeader()`
+<5> and check component-specific messages in `acknowledgeMessages()`.
 
 [[java-rest-high-put-license-async]]
 ==== Asynchronous Execution

docs/reference/aggregations/metrics/tophits-aggregation.asciidoc

@@ -172,7 +172,7 @@ In the example below we search across crawled webpages. For each webpage we stor
 belong to. By defining a `terms` aggregator on the `domain` field we group the result set of webpages by domain. The
 `top_hits` aggregator is then defined as sub-aggregator, so that the top matching hits are collected per bucket.
 
-Also a `max` aggregator is defined which is used by the `terms` aggregator's order feature the return the buckets by
+Also a `max` aggregator is defined which is used by the `terms` aggregator's order feature to return the buckets by
 relevancy order of the most relevant document in a bucket.
 
 [source,js]

docs/reference/cluster/update-settings.asciidoc

@@ -1,9 +1,18 @@
 [[cluster-update-settings]]
 == Cluster Update Settings
 
-Allows to update cluster wide specific settings. Settings updated can
-either be persistent (applied across restarts) or transient (will not
-survive a full cluster restart). Here is an example:
+Use this API to review and change cluster-wide settings. 
+
+To review cluster settings:
+
+[source,js]
+--------------------------------------------------
+GET /_cluster/settings
+--------------------------------------------------
+// CONSOLE
+
+Updates to settings can be persistent, meaning they apply across restarts, or transient, where they don't 
+survive a full cluster restart. Here is an example of a persistent update:
 
 [source,js]
 --------------------------------------------------
@@ -16,7 +25,7 @@ PUT /_cluster/settings
 --------------------------------------------------
 // CONSOLE
 
-Or:
+This update is transient:
 
 [source,js]
 --------------------------------------------------
@@ -29,8 +38,7 @@ PUT /_cluster/settings?flat_settings=true
 --------------------------------------------------
 // CONSOLE
 
-The cluster responds with the settings updated. So the response for the
-last example will be:
+The response to an update returns the changed setting, as in this response to the transient example:
 
 [source,js]
 --------------------------------------------------
@@ -44,11 +52,14 @@ last example will be:
 --------------------------------------------------
 // TESTRESPONSE[s/\.\.\./"acknowledged": true,/]
 
-Resetting persistent or transient settings can be done by assigning a
-`null` value. If a transient setting is reset, the persistent setting
-is applied if available. Otherwise Elasticsearch will fallback to the setting
-defined at the configuration file or, if not existent, to the default
-value. Here is an example:
+You can reset persistent or transient settings by assigning a
+`null` value. If a transient setting is reset, the first one of these values that is defined is applied:
+
+* the persistent setting
+* the setting in the configuration file
+* the default value. 
+
+This example resets a setting:
 
 [source,js]
 --------------------------------------------------
@@ -61,8 +72,7 @@ PUT /_cluster/settings
 --------------------------------------------------
 // CONSOLE
 
-Reset settings will not be included in the cluster response. So
-the response for the last example will be:
+The response does not include settings that have been reset:
 
 [source,js]
 --------------------------------------------------
@@ -74,8 +84,8 @@ the response for the last example will be:
 --------------------------------------------------
 // TESTRESPONSE[s/\.\.\./"acknowledged": true,/]
 
-Settings can also be reset using simple wildcards. For instance to reset
-all dynamic `indices.recovery` setting a prefix can be used:
+You can also reset settings using wildcards. For example, to reset
+all dynamic `indices.recovery` settings:
 
 [source,js]
 --------------------------------------------------
@@ -88,25 +98,19 @@ PUT /_cluster/settings
 --------------------------------------------------
 // CONSOLE
 
-Cluster wide settings can be returned using:
-
-[source,js]
---------------------------------------------------
-GET /_cluster/settings
---------------------------------------------------
-// CONSOLE
 
 [float]
-=== Precedence of settings
+=== Order of Precedence
+
+The order of precedence for cluster settings is:
 
-Transient cluster settings take precedence over persistent cluster settings,
-which take precedence over settings configured in the `elasticsearch.yml`
-config file.
+1. transient cluster settings 
+2. persistent cluster settings
+3. settings in the `elasticsearch.yml` configuration file.
 
-For this reason it is preferrable to  use the `elasticsearch.yml` file only
-for local configurations, and set all cluster-wider settings with the
+It's best to use the `elasticsearch.yml` file only
+for local configurations, and set all cluster-wide settings with the
 `settings` API.
 
-A list of dynamically updatable settings can be found in the
-<<modules,Modules>> documentation.
+You can find the list of settings that you can dynamically update in <<modules,Modules>>.
 

docs/reference/index-modules/store.asciidoc

@@ -67,11 +67,6 @@ process equal to the size of the file being mapped. Before using this
 class, be sure you have allowed plenty of
 <<vm-max-map-count,virtual address space>>.
 
-[[default_fs]]`default_fs` deprecated[5.0.0, The `default_fs` store type is deprecated - use `fs` instead]::
-
-The `default` type is deprecated and is aliased to `fs` for backward
-compatibility.
-
 === Pre-loading data into the file system cache
 
 NOTE: This is an expert setting, the details of which may change in the future.

docs/reference/setup/bootstrap-checks.asciidoc

@@ -118,6 +118,19 @@ least 4096 threads. This can be done via `/etc/security/limits.conf`
 using the `nproc` setting (note that you might have to increase the
 limits for the `root` user too).
 
+=== Max file size check
+
+The segment files that are the components of individual shards and the translog
+generations that are components of the translog can get large (exceeding
+multiple gigabytes). On systems where the max size of files that can be created
+by the Elasticsearch process is limited, this can lead to failed
+writes. Therefore, the safest option here is that the max file size is unlimited
+and that is what the max file size bootstrap check enforces. To pass the max
+file check, you must configure your system to allow the Elasticsearch process
+the ability to write files of unlimited size. This can be done via
+`/etc/security/limits.conf` using the `fsize` setting to `unlimited` (note that
+you might have to increase the limits for the `root` user too).
+
 [[max-size-virtual-memory-check]]
 === Maximum size virtual memory check
 
@@ -133,19 +146,6 @@ address space. This can be done via `/etc/security/limits.conf` using
 the `as` setting to `unlimited` (note that you might have to increase
 the limits for the `root` user too).
 
-=== Max file size check
-
-The segment files that are the components of individual shards and the translog
-generations that are components of the translog can get large (exceeding
-multiple gigabytes). On systems where the max size of files that can be created
-by the Elasticsearch process is limited, this can lead to failed
-writes. Therefore, the safest option here is that the max file size is unlimited
-and that is what the max file size bootstrap check enforces. To pass the max
-file check, you must configure your system to allow the Elasticsearch process
-the ability to write files of unlimited size. This can be done via
-`/etc/security/limits.conf` using the `fsize` setting to `unlimited` (note that
-you might have to increase the limits for the `root` user too).
-
 === Maximum map count check
 
 Continuing from the previous <<max-size-virtual-memory-check,point>>, to

docs/reference/setup/sysconfig/virtual-memory.asciidoc

@@ -1,7 +1,7 @@
 [[vm-max-map-count]]
 === Virtual memory
 
-Elasticsearch uses a <<default_fs,`mmapfs`>> directory by
+Elasticsearch uses a <<mmapfs,`mmapfs`>> directory by
 default to store its indices.  The default operating system limits on mmap
 counts is likely to be too low, which may result in out of memory exceptions.
 

modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceParser.java

@@ -78,7 +78,6 @@ private ArrayValuesSourceParser(boolean formattable, ValuesSourceType valuesSour
         throws IOException {
 
         List<String> fields = null;
-        ValueType valueType = null;
         String format = null;
         Map<String, Object> missingMap = null;
         Map<ParseField, Object> otherOptions = new HashMap<>();
@@ -145,9 +144,6 @@ private ArrayValuesSourceParser(boolean formattable, ValuesSourceType valuesSour
         if (fields != null) {
             factory.fields(fields);
         }
-        if (valueType != null) {
-            factory.valueType(valueType);
-        }
         if (format != null) {
             factory.format(format);
         }

modules/ingest-common/src/test/java/org/elasticsearch/ingest/common/ConvertProcessorTests.java

@@ -19,24 +19,24 @@
 
 package org.elasticsearch.ingest.common;
 
-import org.elasticsearch.ingest.IngestDocument;
-import org.elasticsearch.ingest.Processor;
-import org.elasticsearch.ingest.RandomDocumentPicks;
-import org.elasticsearch.test.ESTestCase;
-
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Locale;
 import java.util.Map;
 
+import org.elasticsearch.ingest.IngestDocument;
+import org.elasticsearch.ingest.Processor;
+import org.elasticsearch.ingest.RandomDocumentPicks;
+import org.elasticsearch.test.ESTestCase;
+
 import static org.elasticsearch.ingest.IngestDocumentMatcher.assertIngestDocument;
 import static org.elasticsearch.ingest.common.ConvertProcessor.Type;
 import static org.hamcrest.Matchers.containsString;
 import static org.hamcrest.Matchers.equalTo;
-import static org.hamcrest.Matchers.sameInstance;
 import static org.hamcrest.Matchers.not;
+import static org.hamcrest.Matchers.sameInstance;
 
 public class ConvertProcessorTests extends ESTestCase {
 
@@ -138,6 +138,7 @@ public void testConvertLongLeadingZero() throws Exception {
         assertThat(ingestDocument.getFieldValue(fieldName, Long.class), equalTo(10L));
     }
 
+    @AwaitsFix( bugUrl = "https://github.com/elastic/elasticsearch/issues/32370")
     public void testConvertLongHexError() {
         IngestDocument ingestDocument = RandomDocumentPicks.randomIngestDocument(random());
         String value = "0x" + randomAlphaOfLengthBetween(1, 10);

modules/lang-expression/licenses/lucene-expressions-7.5.0-snapshot-608f0277b0.jar.sha1

@@ -0,0 +1 @@
+bd7d8078a2d0ad11a24f54156cc015630c96858a

modules/lang-painless/spi/src/main/java/org/elasticsearch/painless/spi/PainlessExtension.java

@@ -19,11 +19,11 @@
 
 package org.elasticsearch.painless.spi;
 
+import org.elasticsearch.script.ScriptContext;
+
 import java.util.List;
 import java.util.Map;
 
-import org.elasticsearch.script.ScriptContext;
-
 public interface PainlessExtension {
 
     Map<ScriptContext<?>, List<Whitelist>> getContextWhitelists();

modules/lang-painless/src/main/java/org/elasticsearch/painless/Def.java

@@ -421,7 +421,7 @@ static MethodHandle lookupGetter(PainlessLookup painlessLookup, Class<?> receive
             PainlessClass struct = painlessLookup.getPainlessStructFromJavaClass(clazz);
 
             if (struct != null) {
-                MethodHandle handle = struct.getters.get(name);
+                MethodHandle handle = struct.getterMethodHandles.get(name);
                 if (handle != null) {
                     return handle;
                 }
@@ -431,7 +431,7 @@ static MethodHandle lookupGetter(PainlessLookup painlessLookup, Class<?> receive
                 struct = painlessLookup.getPainlessStructFromJavaClass(iface);
 
                 if (struct != null) {
-                    MethodHandle handle = struct.getters.get(name);
+                    MethodHandle handle = struct.getterMethodHandles.get(name);
                     if (handle != null) {
                         return handle;
                     }
@@ -492,7 +492,7 @@ static MethodHandle lookupSetter(PainlessLookup painlessLookup, Class<?> receive
             PainlessClass struct = painlessLookup.getPainlessStructFromJavaClass(clazz);
 
             if (struct != null) {
-                MethodHandle handle = struct.setters.get(name);
+                MethodHandle handle = struct.setterMethodHandles.get(name);
                 if (handle != null) {
                     return handle;
                 }
@@ -502,7 +502,7 @@ static MethodHandle lookupSetter(PainlessLookup painlessLookup, Class<?> receive
                 struct = painlessLookup.getPainlessStructFromJavaClass(iface);
 
                 if (struct != null) {
-                    MethodHandle handle = struct.setters.get(name);
+                    MethodHandle handle = struct.setterMethodHandles.get(name);
                     if (handle != null) {
                         return handle;
                     }

modules/lang-painless/src/main/java/org/elasticsearch/painless/PainlessExplainError.java

@@ -22,6 +22,7 @@
 import org.elasticsearch.painless.api.Debug;
 import org.elasticsearch.painless.lookup.PainlessClass;
 import org.elasticsearch.painless.lookup.PainlessLookup;
+import org.elasticsearch.painless.lookup.PainlessLookupUtility;
 import org.elasticsearch.script.ScriptException;
 
 import java.util.List;
@@ -58,7 +59,7 @@ public Map<String, List<String>> getHeaders(PainlessLookup painlessLookup) {
             javaClassName = objectToExplain.getClass().getName();
             PainlessClass struct = painlessLookup.getPainlessStructFromJavaClass(objectToExplain.getClass());
             if (struct != null) {
-                painlessClassName = struct.name;
+                painlessClassName = PainlessLookupUtility.typeToCanonicalTypeName(objectToExplain.getClass());
             }
         }