Self monitoring stops uploading if connection to Elasticsearch is lost

[joshdover]

Kibana version: 7.11 & master

Elasticsearch version: 7.11 & master

Server OS version:

Browser version:

Browser OS version:

Original install method (e.g. download page, yum, from source, etc.):

Describe the bug:

If self-monitoring is used and Kibana loses its connection to Elasticsearch, it does not resume reporting metrics once the connection is reestablished.

Steps to reproduce:

1. Start Kibana and Elasticsearch
2. Enable self-monitoring via the UI
3. Stop Elasticsearch
4. Wait for error logs:

   Unable to bulk upload the stats payload to the local cluster
   Error: Cluster client cannot be used after it has been closed.
   

5. Start Elasticsearch
6. Same error logs repeat and monitoring data stops flowing
7. Restarting Kibana resolves the issue

Expected behavior:

Self-reported monitoring should be able to recover from connection failures

Screenshots (if relevant):

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

log   [08:39:39.864] [debug][kibana-monitoring][monitoring][monitoring][plugins] Uploading bulk stats payload to the local cluster
  log   [08:39:39.866] [warning][kibana-monitoring][monitoring][monitoring][plugins] Error: Cluster client cannot be used after it has been closed.
    at LegacyClusterClient.assertIsNotClosed (/usr/local/Cellar/kibana-full/7.11.2/libexec/src/core/server/elasticsearch/legacy/cluster_client.js:195:13)
    at LegacyClusterClient.callAsInternalUser (/usr/local/Cellar/kibana-full/7.11.2/libexec/src/core/server/elasticsearch/legacy/cluster_client.js:115:12)
    at sendBulkPayload (/usr/local/Cellar/kibana-full/7.11.2/libexec/x-pack/plugins/monitoring/server/kibana_monitoring/lib/send_bulk_payload.js:22:18)
    at BulkUploader._onPayload (/usr/local/Cellar/kibana-full/7.11.2/libexec/x-pack/plugins/monitoring/server/kibana_monitoring/bulk_uploader.js:209:43)
    at BulkUploader._fetchAndUpload (/usr/local/Cellar/kibana-full/7.11.2/libexec/x-pack/plugins/monitoring/server/kibana_monitoring/bulk_uploader.js:195:20)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
  log   [08:39:39.866] [warning][kibana-monitoring][monitoring][monitoring][plugins] Unable to bulk upload the stats payload to the local cluster

Any additional context:

I tested this scenario in ELK 7.10.2 and could not reproduce the issue so this does appear to be a regression in 7.11+. I did notice this that this refactor PR touched this area of the code: #83546

[elasticmachine]

Pinging @elastic/stack-monitoring (Team:Monitoring)