switch http client to something that does not have a native extension

[grosser]

faraday or httparty would come to my mind ...

 gem i rest-client
Fetching: unf_ext-0.0.7.2.gem (100%)
Building native extensions.  This could take a while...
Successfully installed unf_ext-0.0.7.2
Fetching: unf-0.1.4.gem (100%)
Successfully installed unf-0.1.4
Fetching: domain_name-0.5.20170223.gem (100%)
Successfully installed domain_name-0.5.20170223
Fetching: http-cookie-1.0.3.gem (100%)
Successfully installed http-cookie-1.0.3
Fetching: mime-types-data-3.2016.0521.gem (100%)
Successfully installed mime-types-data-3.2016.0521
Fetching: mime-types-3.1.gem (100%)
Successfully installed mime-types-3.1
Fetching: netrc-0.11.0.gem (100%)
Successfully installed netrc-0.11.0
Fetching: rest-client-2.0.1.gem (100%)
Successfully installed rest-client-2.0.1
8 gems installed

[cben]

Oh, I thought Kubeclient is purely rest-client but turns out we also use http gem for WatchStream functionality.
And http also requires native extenstions via http-cookie < domain_name < unf < unf_ext (EDIT: not nokogiri that's, only for devel).

I propose any replacement should:

• support streaming so it can replace both rest-client & http.
• use HTTP persistent connections. rest-client opens new socket (and TLS handshake) for every request :-( Connection reuse rest-client/rest-client#453

And old but detailed comparison: https://bibwild.wordpress.com/2012/04/30/ruby-http-performance-shootout-redux/
=> httpclient gem came on top there, pure ruby (zero deps 🥇), has persistent connections + thread safe. TOCHECK:

• rerun those benchmarks, there is a newer fork but also needs some updating
• does httpclient support GET streaming? get_content yields chunks of String, get_async returns HTTPClient::Connection — sounds good on paper.

https://jvns.ca/blog/2016/03/04/whats-up-with-ruby-http-libraries/ sumarises nahi's epic table https://bit.ly/RubyHTTPClients2012 (slides), with a crucial point that at the bottom everything builds on just 4 libs: Net:HTTP, Excon, httpclient and native libcurl.
IMHO adopting one well-designed lib sounds nicer than a stack of several...

Faraday is unusual in supporting many backend libs with same API.

• Faraday can use Patron, a native libcurl binding. But it seems it doesn't automatically pull it in, so that's OK.
• It seems streaming could only work with Net::HTTP, and hasn't landed yet: Streaming requests for Net::HTTP lostisland/faraday#604. Oh, @grosser you reviewed that — can you say if we'd need it to land before we could do watching?

[cben]

httparty is also pure ruby, uses 'net/http'.

• Streaming: looks like yes: https://github.com/jnunemaker/httparty/blob/master/examples/stream_download.rb
• No builtin persistent (Interested in support for persistent http connections? jnunemaker/httparty#155) but there is https://github.com/soupmatt/persistent_httparty

[grosser]

Great summary! :D Faraday is in a weird state maintainer wise ... so not sure if that will work out ... so far httpclient sounds like the winner to me too ... just need some benchmarks :)

…

On Sun, Mar 26, 2017 at 3:17 AM, Beni Cherniavsky-Paskin < ***@***.***> wrote: httparty is also pure ruby, uses 'net/http'. - Streaming: looks like yes: https://github.com/jnunemaker/ httparty/blob/master/examples/stream_download.rb <https://github.com/jnunemaker/httparty/blob/master/examples/stream_download.rb> - No builtin persistent (jnunemaker/httparty#155 <jnunemaker/httparty#155>) but there is https://github.com/soupmatt/persistent_httparty <https://github.com/soupmatt/persistent_httparty> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#237 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAsZwjs3ue0jISv2bibQfY0Ff3gU0Yqks5rpjtGgaJpZM4Mde3k> .

[cben]

@msufa You'd switched watching to use http.rb for Celluloid support — can you comment whether any of the alternatives here (especially httpclient) could work for your needs?
(BTW, do you know if we still need http.rb pegged to 0.9.8? It's now at 2.2.1, with 3.x branch in the works. A dependency we can't upgrade is one more reason to consider a switch...)

---

Other requirements: #140 lists several.
I'd also like to understand where kubernetes API is going — protobuf?
HTTP/2? gRPC? @smarterclayton can you give us a hint what a k8s client lib should plan for?

[msufa]

@cben as noted in #204 (comment) Celluloid support should no longer be necessary, so you're free to go with whichever HTTP implementation you find works best for you :)

[moolitayer]

see #254

[cben]

We might also need websocket support at some point. No idea what means in terms of gems.

It seems from docs websockets are already supported for watches (don't know if it gives any benefit over GET), and looks like they'll be required for certain types of bulk watching: kubernetes/community#443

[grosser]

FYI faraday now has streaming lostisland/faraday#604

[cben]

Just a data point: replacing http and rest-client with httpclient would make the bundle way smaller: 34MB -> 5MB. It's now dominated by ffi and unf_ext.

[andrzej-stencel]

Folks, here's draft PR (work in progress): #466 to give users the option to replace rest_client with httpclient with an initialization parameter like this:

client = Kubeclient::Client.new('https://apiserver/api', 'v1', http_client_type: 'httpclient')

I'd love to hear your thoughts on this.

[cben]

In case anyone here has opinions between Faraday / httpclient / another lib — NOW is your last chance to influence! 😉 Join #466 discussion ^^. Note that support for connection reuse is a must for @astencel-sumo.