Backport latest doc changes from master to 1.0 (#2420)

* Add beta -> upgrade docs (#2407)

* Add release highlights (#2356)

Co-authored-by: Anya Sabo <1638148+anyasabo@users.noreply.github.com>

docs/release-notes/1.0.0.asciidoc

@@ -8,8 +8,7 @@
 [float]
 === Breaking changes
 
-* Remove v1alpha1 CRD version & generate a single trivial all-in-one flavor {pull}2119[#2119] (issue: {issue}2044[#2044])
-* Add multi-namespace cache support {pull}1995[#1995]
+* Remove v1alpha1 CRD version and generate a single trivial all-in-one flavor {pull}2119[#2119] (issue: {issue}2044[#2044])
 
 
 [[feature-1.0.0]]
@@ -22,18 +21,26 @@
 [float]
 === Enhancements
 
-* Support 7.6 and new stack level enterprise license type. {pull}2242[#2242]
-* Minimize downtime during pod recycling {pull}2233[#2233] (issue: {issue}1927[#1927])
+* Report max ERUs in the licensing info configmap {pull}2371[#2371]
+* Readiness probe: do not log tail errors {pull}2366[#2366]
+* Wait 30sec in the preStop hook to account for kube-proxy refresh {pull}2360[#2360] (issue: {issue}2318[#2318])
+* Allow user to override readiness timeout {pull}2260[#2260] (issue: {issue}2259[#2259])
+* Support 7.6 and new stack level enterprise license type {pull}2242[#2242]
+* Minimize downtime during Pod recycling {pull}2233[#2233] (issue: {issue}1927[#1927])
+* Bump ctrl-runtime dependency {pull}2232[#2232]
 * Do not reconcile APM Server if association is not fully configured {pull}2224[#2224]
-* Upgrade from trial to platinum/enterprise license {pull}2206[#2206]
+* Upgrade from trial to Platinum/Enterprise license {pull}2206[#2206]
 * Remove finalizers {pull}2195[#2195]
-* Downgrade to basic if user deletes license secret {pull}2188[#2188]
-* Added limits to initpod {pull}2186[#2186] (issue: {issue}2179[#2179])
+* Downgrade to Basic if user deletes license secret {pull}2188[#2188]
+* Add default requests and limits to the init containers {pull}2186[#2186] (issue: {issue}2179[#2179])
+* Sort seed hosts to avoid unecessary configmap updates {pull}2171[#2171]
 * Turn blacklist into warning events and logs statements {pull}2162[#2162]
 * Autostart trial {pull}2160[#2160]
 * Reconcile all clusters on license change {pull}2145[#2145]
 * Pods upgrade: log a summary of failed predicates {pull}2128[#2128]
-* Add webhook Secret and ValidatingWebhookConfiguration certificate management {pull}2126[#2126]
+* Add webhook secret and ValidatingWebhookConfiguration certificate management {pull}2126[#2126]
+* Name service ports based on protocol {pull}2083[#2083]
+* Simplify license installation {pull}2073[#2073]
 * Check resource version when deleting a Pod during force-upgrades {pull}2066[#2066] (issue: {issue}1916[#1916])
 * Perform forced rolling upgrade even if ES is reachable {pull}2022[#2022]
 * Refactor expectations with proper garbage collection {pull}2000[#2000] (issue: {issue}1611[#1611])
@@ -43,6 +50,14 @@
 [float]
 === Bug fixes
 
+* Do not report max ERUs for basic licenses {pull}2377[#2377]
+* Fix v1beta1 webhook {pull}2358[#2358] (issue: {issue}2355[#2355])
+* Preserve environment variable order in Pod specification {pull}2341[#2341]
+* Reuse the same upscaleState across StatefulSets {pull}2339[#2339] (issue: {issue}2338[#2338])
+* Allow node restart even if cluster health is yellow {pull}2330[#2330]
+* Do not upgrade Pods if empty StatefulSet UpdateRevision {pull}2321[#2321] (issue: {issue}2320[#2320])
+* Fix how cluster.initial_master_nodes is set {pull}2315[#2315] (issue: {issue}2291[#2291])
+* Wait for webhook key to be present in filesystem {pull}2312[#2312]
 * Do not fail if annotations file does not exist {pull}2275[#2275]
 * Fix readiness probe {pull}2272[#2272]
 * Change priority order of reconcile results {pull}2250[#2250]
@@ -54,12 +69,19 @@
 * Update service when labels and annotations are modified {pull}2210[#2210]
 * Fix readiness script in case of operator upgrade {pull}2208[#2208]
 * Restore v1alpha1 in the list of crds {pull}2199[#2199] (issue: {issue}2196[#2196])
-* Don't use env variables ending in _FILE with Elasticsearch {pull}2180[#2180]
+* Do not use env variables ending in _FILE with Elasticsearch {pull}2180[#2180]
 * Ignore and do not use an empty CA {pull}2140[#2140]
-* Fix Kibana to terminate all pods before restarting during version change {pull}2137[#2137]
+* Fix Kibana to terminate all Pods before restarting during version change {pull}2137[#2137]
+* Perform StatefulSets deletions before replicas downscale {pull}2062[#2062]
 * Always enable native realm {pull}2038[#2038]
 * Fix nil pointer in upgrade predicates {pull}2035[#2035]
 * Use discovery.seed_providers instead of discovery.zen.hosts_provider starting 7.x {pull}2029[#2029]
 * Make association optional for Kibana {pull}2021[#2021]
 * Fix result of the APM Server controller  {pull}1991[#1991]
 * Mitigate memory leaks from long RequeueAfter periods {pull}1989[#1989]
+
+[[nogroup-1.0.0]]
+[float]
+=== Misc
+
+* Change rolling upgrades predicate log error to info level {pull}2099[#2099]

docs/release-notes/highlights-1.0.0-beta1.asciidoc

@@ -44,7 +44,7 @@ Default limits are now set for the various resource types if none are explicitly
 === Breaking Changes
 
 The 1.0.0-beta version of the operator does not delete resources created by older versions of the operator, but it also does not manage them. 
-Attempting to delete resources created with a 0.9.0 version will hang if ECK 1.0.0-beta1 is running. Please see the <<{p}-1.0.0-beta1-backwards-compatibility, upgrading documentation>> for more information.
+Attempting to delete resources created with a 0.9.0 version will hang if ECK 1.0.0-beta1 is running. Please see the <<{p}-upgrading-eck, upgrading documentation>> for more information.
 
 [float]
 [id="{p}-known-issues-1.0.0-beta1"]

docs/release-notes/highlights-1.0.0.asciidoc

@@ -0,0 +1,38 @@
+[[release-highlights-1.0.0]]
+== 1.0.0 release highlights
+
+[float]
+[id="{p}-general-availability"]
+=== General availability
+
+Elastic Cloud on Kubernetes has graduated from beta and is now in general availability.
+
+[float]
+[id="{p}-release-v1"]
+=== Resource version promotion
+
+Custom resources have graduated to version `v1`.  Manifests for `v1beta1` will still function, but are deprecated and should be updated to `v1`. Support for `v1beta1` will be removed in a future release. Support for `v1alpha1` has been removed. See <<{p}-upgrading-eck>> for more information.
+
+[float]
+[id="{p}-release-webhook"]
+=== Webhook validation
+
+Webhook validation has been re-introduced. This allows for additional validation when Elastic resources are created or updated. See <<{p}-webhook>> for more information.
+
+[float]
+[id="{p}-multi-namespace"]
+=== Multiple namespace management
+
+ECK can now manage multiple specific namespaces simultaneously. Previously a single instance of ECK could manage either all namespaces or a single namespace. See <<{p}-operator-config>> for more information.
+
+[float]
+[id="{p}-release-license-mgmt"]
+=== License management
+
+Handling of licenses has been greatly improved. If the license changes, all new clusters will use it and all existing clusters will be upgraded to the new license. See <<{p}-licensing>> for more information.
+
+[float]
+[id="{p}-release-mesh-compat"]
+=== Service mesh compatibility
+
+Compatibility with service mesh systems such as Istio has been improved, with changes such as link:https://github.com/elastic/cloud-on-k8s/pull/2083[#2038].

docs/release-notes/highlights.asciidoc

@@ -6,8 +6,10 @@
 This section summarizes the most important changes in each release. For the
 full list, see <<eck-release-notes>>.
 
+* <<release-highlights-1.0.0>>
 * <<release-highlights-1.0.0-beta1>>
 
 --
 
+include::highlights-1.0.0.asciidoc[]
 include::highlights-1.0.0-beta1.asciidoc[]

docs/upgrading-eck.asciidoc

@@ -7,131 +7,27 @@ endif::[]
 == Upgrading ECK
 
 [float]
-[id="{p}-upgrade-to-v1beta1"]
-=== Upgrading to ECK 1.0.0-beta1 from previous versions
+[id="{p}-ga-upgrade"]
+=== Upgrading to ECK 1.0.0 from previous versions
 
-ECK 1.0.0-beta1 includes changes that are incompatible with previous versions of the operator. Notable changes include:
+ECK 1.0.0 is largely compatible with the beta version of the operator (see <<{p}-ga-openshift,the exceptions for earlier Kubernetes and OpenShift versions>>). There is not a direct upgrade path from the alpha version.
 
-- Custom resource version has changed from `v1alpha1` to `v1beta1`.
-- Some CRD fields have been removed and some others have been renamed to clarify their purpose.
-- Elasticsearch cluster orchestration is now managed through link:https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/[StatefulSet] resources.
+To upgrade from 1.0.0-beta1, follow the <<{p}-quickstart>>. Note that all Elasticsearch Pods go through a rolling restart when the operator is upgraded.
 
-NOTE: See the <<release-notes-1.0.0-beta1>> for more information about all the features included in this release.
+After the upgrade, use the `v1` API version for all Elastic manifests.
 
-It is recommended to install ECK 1.0.0-beta1 in a fresh Kubernetes cluster and migrate data over from existing clusters if possible. If you wish to install ECK into an existing Kubernetes cluster that has a previous version of the operator installed, it is important to consider the following:
-
-- The old operator will be replaced by the new operator during the installation process.
-- Existing Elasticsearch, Kibana and APM Server resources created by an old version of the operator will continue to work but they will not be managed by the new operator. This means that the orchestration benefits provided by the operator such as rolling upgrades will no longer be available to those resources.
-- If the old operator is replaced without removing old resources first, you will have to manually disable finalizers to delete them later.
-- Existing Kubernetes manifests should be converted to the new format in order to work with the new operator.
-- link:https://github.com/elastic/cloud-on-k8s/issues/2039[Some incompatible settings] in Elasticsearch resources created for an older ECK version lead to parsing errors in ECK 1.0.0-beta1, preventing any reconciliation from happening.
-
-If some downtime is acceptable, upgrading in place can be performed as follows:
-
-CAUTION: These instructions are general guidelines only. You should have a thoroughly tested upgrade plan for your environment before attempting to modify any production deployments.
-
-. <<{p}-convert-manifests,Convert existing manifests to the new format>>.
-. link:https://www.elastic.co/guide/en/cloud-on-k8s/0.9/k8s-snapshot.html[Create snapshots of your existing Elasticsearch clusters].
-. link:https://www.elastic.co/guide/en/cloud-on-k8s/0.9/k8s-uninstall.html[Uninstall ECK].
-. link:https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html[Install new version of ECK].
-. Re-create the resources by applying the manifests converted in the first step.
-. Restore data from the snapshots.
+See the <<release-highlights-1.0.0>> for more information on new features.
 
 [float]
-[id="{p}-convert-manifests"]
-==== Convert Manifests
-
-.Elasticsearch
-* Replace `v1alpha1` in the `apiVersion` field with `v1beta1`
-* Rename `nodes` to `nodeSets`
-* Ensure that every `nodeSets` entry has a `name` and rename `nodeCount` to `count`
-* Remove `setVmMaxMapCount`. See: <<{p}-virtual-memory>>.
-* Remove `groups` from `updateStrategy`. See: <<{p}-update-strategy>>.
-* Remove `featureFlags`
-* If you specified `secureSettings`, convert `secretName` into an array item by prefixing it with `-`
-
-[source,patch,subs="attributes"]
-----
--apiVersion: elasticsearch.k8s.elastic.co/v1alpha1
-+apiVersion: elasticsearch.k8s.elastic.co/v1beta1
- kind: Elasticsearch
- metadata:
-   name: elasticsearch-sample
- spec:
-   version: {version}
--  setVmMaxMapCount: true
--  featureFlags: {}
-   updateStrategy:
-     changeBudget:
-       maxUnavailable: 1
--    groups: []
-   secureSettings:
--    secretName: "gcs-credentials"
-+  - secretName: "gcs-credentials"
--  nodes:
--  - nodeCount: 3
-+  nodeSets:
-+  - count: 3
-+    name: master-nodes
-----
+[id="{p}-ga-openshift"]
+=== Upgrades in earlier Kubernetes and OpenShift versions
 
+Upgrading from ECK 1.0.0-beta1 to 1.0.0 has some special considerations for users running either:
+- Kubernetes versions < 1.13
+- OpenShift 3.11
 
-.Kibana
-* Replace `v1alpha` in the `apiVersion` field with `v1beta1`
-* Rename `nodeCount` to `count`
-* If you specified `secureSettings`, convert `secretName` into an array item by prefixing it with `-`
-
-[source,patch,subs="attributes"]
-----
--apiVersion: kibana.k8s.elastic.co/v1alpha1
-+apiVersion: kibana.k8s.elastic.co/v1beta1
- kind: Kibana
- metadata:
-   name: kibana-sample
- spec:
-   version: {version}
--  nodeCount: 1
-+  count: 1
-   elasticsearchRef:
-     name: "elasticsearch-sample"
-   secureSettings:
--    secretName: "gcs-credentials"
-+  - secretName: "gcs-credentials"
-----
-
-
-.APM Server
-* Replace `v1alpha` in the `apiVersion` field with `v1beta1`
-* Rename `nodeCount` to `count`
-* If you specified `secureSettings`, convert `secretName` into an array item by prefixing it with `-`
-
-[source,patch,subs="attributes"]
-----
--apiVersion: apm.k8s.elastic.co/v1alpha1
-+apiVersion: apm.k8s.elastic.co/v1beta1
- kind: ApmServer
- metadata:
-   name: apm-server-sample
- spec:
-   version: {version}
--  nodeCount: 1
-+  count: 1
-   elasticsearchRef:
-     name: "elasticsearch-sample"
-   secureSettings:
--    secretName: "gcs-credentials"
-+  - secretName: "gcs-credentials"
-----
-
-[float]
-[id="{p}-1.0.0-beta1-backwards-compatibility"]
-==== Backwards compatibility
+In this case, ECK must be uninstalled and reinstalled (due to link:https://github.com/kubernetes/kubernetes/issues/73752[an upstream bug] that was not backported). To uninstall ECK, see <<{p}-uninstalling-eck>>. Then you can follow the <<{p}-quickstart>> to install anew. You may wish to save your existing `v1beta1` manifests and update them to the `v1` API version, and to <<{p}-snapshots,take snapshots>> and restore them as well.
 
-The 1.0.0-beta version of the operator does not delete resources created by older versions of the operator, but it also does not manage them. Attempting to delete resources created with a 0.9.0 version will hang if ECK 1.0.0-beta1 is running. To unblock the deletion, remove any registered finalizer from the resource (substituting the correct name for `quickstart`):
+In this case, ECK must be uninstalled and reinstalled (due to link:https://github.com/kubernetes/kubernetes/issues/73752[an upstream bug] that was not backported). To uninstall ECK, see <<{p}-uninstalling-eck>>. Then you can follow the <<{p}-quickstart>> to install anew. You can save your existing `v1beta1` manifests, update them to the `v1` API version, <<{p}-snapshots,take snapshots>> and restore them as well.
 
-[source,sh]
-----
-kubectl patch elasticsearch quickstart --patch '{"metadata": {"finalizers": []}}' --type=merge
-kubectl patch kibana quickstart --patch '{"metadata": {"finalizers": []}}' --type=merge
-kubectl patch apmserver quickstart --patch '{"metadata": {"finalizers": []}}' --type=merge
-----
+After the upgrade, use the `v1` API version for all Elastic manifests.