add option to limit grpc logging length

[leiyiz]

What type of PR is this?
/kind feature

What this PR does / why we need it:
in some cases the GRPC response is huge and logging it entirely causes log space to fill up immediately.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Now there's an option to set a universal limit on GRPC log length.

[leiyiz]

this is similar to how pdcsi driver is going to limit log length in kubernetes-sigs/gcp-compute-persistent-disk-csi-driver#1085

[leiyiz]

/assign @msau42

[msau42]

Log length is mainly an issue with List volume and snapshot calls, where there could be a lot of objects in the system.

The main downside I can see with this approach is that if there are any important fields in the struct after the list of volumes/snapshots, then those will get dropped. But in practice, that is not relevant to the csi list calls.

Another approach could be to allow setting a custom log function, and then having attacher and snapshotter implement some custom handling where maybe it could treat List calls at a debug log level. WDYT? Do you see value in logging the first few entries vs not logging any entries at all?

[leiyiz]

I think with truncated logging we can potentially do some driver-side randomization on the sequence of the list of Volumes in the future thus retain some debuggability. With the custom logger approach my intuition is that there's more potential to accidentally blast other types of rpc call also.

[msau42]

Thanks, I think this is sufficient for now. If anyone else thinks a custom logger may be beneficial, we can look into that as another enhancement.

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: leiyiz, msau42

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [msau42]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment