use better defaults for proxy-next-upstream(-tries)

[ElvinEfendi]

What this PR does / why we need it:
By default ingress-nginx configures Nginx to retry next upstream indefinitely for any of error timeout invalid_header http_502 http_503 http_504. This is a bit extreme and it potentially hides important errors. The PR changes the defaults to be more restrictive. Note that these values can still be customized per ingress.

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

[ElvinEfendi]

/assign @aledbf

[aledbf]

@ElvinEfendi not sure about removing http_502 http_503.

[ElvinEfendi]

@aledbf I'd have expected this to default to what Nginx has as default value: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream which is error timeout.

For example it is a common practice to return 503 during maintenance (for SEO purposes returning 503 is crucial) which does not make sense to retry.

Users can customize the value later if they need to, both through configmap as well as through Ingress annotation.

[aledbf]

@ElvinEfendi I don't share your opinion but to be consistent I think we need to provide the same default values

[aledbf]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aledbf, ElvinEfendi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [aledbf]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment