Support API chunking

[cben]

k8s recently finally got response chunking (kinda pagination, of a consistent snapshot at single resourceVersion!):
kubernetes/community#896 => https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/api-chunking.md
kubernetes/enhancements#365
https://kubernetes.io/docs/reference/using-api/api-concepts/

This has similar benefits to streaming parsing (#254) for peak RAM on client, but also reduces peak RAM on server, and avoids some annoyances like client timing out while server builds response.
Would be great in at some point in kubeclient.
Do we still also want streaming parsing? Don't have an opinion yet.

First I want to figure out if same block-based API as in #254 is also good for chunking? Or do we need something lower-level that gives user more control?

[cben]

Seems this is supported starting with kubernetes 1.8 API.

[cben]

2 open questions:

1. How should whole-collection metadata, notably resource_version, be returned?
   Process JSON Body of ListObjects using streaming json parser #254 in current form first yields each item, at the end returns a hash with resourceVersion & kind keys.
   For streaming parse this was motivated by not assuming k8s always puts items key last in the JSON.
   For chunking, we do have metadata in each chunk, and parse whole chunk at a time, so could also expose it earlier.

2. what to do with 410 Gone mid-iteration?

   Like a watch operation, a continue token will expire after a short amount of time (by default 5 minutes) and return a 410 Gone if more results cannot be returned. In this case, the client will need to start from the beginning or omit the limit parameter.
   -- api-concepts

   How should this scenario be handled by kubeclient? Raise exception? Does this preclude access to whole-collection metadata?

Perhaps we need a 2-step interface like watch:

iter = client.iter_pods
puts "can access #{iter.resourceVersion} early?"
begin
  iter.each { |pod| ... }
  puts "can access #{iter.resourceVersion} at the end"
rescue
  puts "can still access #{iter.resourceVersion} if lost it mid-way!"
end

robust but awkward :-(
would prefer simpler 1-step client.get_pods { ... } if possible.

[agrare]

@cben I took a crack at this over in #356
Not sure that it is the interface that you're looking for but it works well, see full example usage in https://github.com/agrare/openshift-collector/blob/wip_chunk_kubernetes_api_responses/lib/openshift/collector.rb#L85-L113