Implement retry functionality?

[atc0005]

Similar to what we're doing in the go-teams-notify project, allow user-specified number of retries using a user-specified delay between retry attempts.

It will be important to get the default values setup so that the total attempts falls under the default plugin timeout value used by Nagios, Icinga, etc.

[atc0005]

With the DUO auth issues today (https://status.duo.com/incidents/rw7g0q7ztj8f) I'm seeing a lot of flapping between UNKNOWN and WARNING states.

The UNKNOWN state notifications have this output:

Info:

UNKNOWN: Failed to process JSON feed from URL: failed to decode JSON data

Additional Info:

ERRORS

• task: decode JSON data: failed to decode JSON data: source: https://status.duo.com/api/v2/components.json cause: failed to decode JSON feed from source https://status.duo.com/api/v2/components.json: context deadline exceeded

The upstream https://metastatuspage.com/ Statuspage dashboard does not indicate any problems, so without digging deeper it is unclear whether it is an issue with the Nagios "console" or timeouts for the status.duo.com site/API.

Not longer after I filed this GH issue I realized that implementing a retry directly in the plugin was perhaps not needed. We could instead set the retry behavior within the service check before declaring a state change.

I'm divided on that point; added some additional context as noted above for further consideration.