proposal: freeze net/rpc

[robpike]

The package has outstanding bugs that are hard to fix, and cannot support TLS without major work. So although it has a nice API and allows one to use native Go types without an IDL, it should probably be retired.

The proposal is to freeze the package, retire the many bugs filed against it, and add documentation indicating that it is frozen and that suggests alternatives such as GRPC.

[mattn]

retiring jsonrpc too?

[bradfitz]

No. JSON-RPC is at least kinda an external spec. Let's not conflate the two here.

[rhedile]

Whats the problem with TLS? Been using the Package with TLS and jsonrpc for
18 months for internal distribution without problems, with this recipe
https://gist.github.com/artyom/6897140

Could someone please formalise the term "freeze". the suggested link into
the wiki returns "not found"

On 23 August 2016 at 02:28, Rob Pike notifications@github.com wrote:

The package has outstanding bugs that are hard to fix, and cannot support
TLS without major work. So although it has a nice API and allows one to use
native Go types without an IDL, it should probably be retired.

The proposal is to freeze the package, retire the many bugs filed against
it, and add documentation indicating that it is frozen and that suggests
alternatives such as GRPC.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#16844, or mute the thread
https://github.com/notifications/unsubscribe-auth/AEAf7NjnEUYC1RUdPjP11jWSKgbK1EKsks5qij6ygaJpZM4JqcDQ
.

[dmitshur]

I'm in favor of the proposal for the following reason,

The package has outstanding bugs that are hard to fix, and cannot support TLS without major work.

Also because the standard library where its API cannot evolve is not a good place for it.

But I think the value of the following fact is significantly understated.

allows one to use native Go types without an IDL

So I think that grpc is not a viable alternative for anyone who wants to preserve that property. It's a fine suggestion for many.

But, a more net/rpc-like package with support for native Go types would be better developed (by those who are interested) outside of the std lib tree, where it's API could improve to support context.Context, etc.

[peterbourgon]

I agree with @shurcooL that the value of using Go native types without an IDL is very high, even if just for introductory and prototype purposes. I would hate to see net/rpc removed because it was difficult to satisfy some sophisticated use cases. Maybe it can just be frozen, and appropriate warnings issued in the package documentation?

edit: I misinterpreted retired as removed; mea culpa

[bradfitz]

Nobody is proposing violating the Go 1 compatibility promise and removing anything.

[robpike]

Exactly. As I wrote, "the proposal is to freeze the package".

[adnaan]

The following anecdotes of my experience with net/rpc and GRPC only makes sense if my understanding that freezing net/rpc would impact the growth of net/rpc/jsonrpc is correct? If not, I just wasted 15 minutes writing this. Here goes:

I have worked with both net/rpc/jsonrpc and grpc in my projects. Incidentally both use cases were: Go server <-> JSON RPC Java Client. The first use case was to net/rpc/jsonrpc on localhost for automation testing of a phone (uiautomator) and second was a proper Android app using GRPC's streaming capabilities.

In my experience, setting up net/rpc/jsonrpc was trivial, and there was enough Java JSONRPC support to be found while GRPC took a bit of work setting up.

1. Firstly you need to premeditate your API design in the IDL which does sounds nice but is a pain if you want to get a prototype done quickly.
2. Even though I was able to setup TLS on Android, till day I am not sure whether the setup was right since the documentation is lacking(checked again, the documentation is in the same state).
3. I had to write "weird" extra code to utilize the blockingStub in an executor pool.

I guess points 2 & 3 stem from the fact that there hasn't been enough work on GRPC to form a standard body of work. Also I realise 2 & 3 are client issues but when you choose GRPC the client libraries get chosen for you by default.

net/rpc/jsonrpc provides a lot of flexibility of choice while picking a client library and is quicker and easier to use.

[bradfitz]

The following anecdotes of my experience with net/rpc and GRPC only makes sense if my understanding that freezing net/rpc would impact the growth of net/rpc/jsonrpc is correct?

False. See the first two replies in this thread.

[sbunce]

SGTM on freezing net/rpc.

The net/rpc code is nice and simple. The interfaces are well designed. I use net/rpc with proto codecs to avoid the complexity and dependencies of grpc. Like rhedile mentioned TLS can be used without much difficulty. Freezing doesn't effect these use cases.

[fwessels]

We are currently using net/rpc in a.o. https://github.com/minio/minio and https://github.com/minio/dsync and are pretty happy with it, but are open to switching (and getting better performance as a bonus).

Are there any performance comparisons between net/rpc and grpc? Also Golang is (strangely?) missing from the benchmarking results as per eg https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5712453606309888&widget=828849126&container=1808124444&maximized

(BTW there are huge gaps between C++ at 250K QPS/sec vs Java at 150K QPS/sec vs C# at 20K QPS/sec -- especially the last two I would expect to be similar. And these are all async calls, in our case we'd be using synchronous calls).

[fwessels]

We've done (using https://github.com/minio/dsync) some performance comparisons between net/rpc and grpc and we’ve been surprised to find that grpc is slower by about 4x (given same CPU load).

The following table lists the results of a test with 8 EC2 servers that shows that grpc gives roughly half the results at double the CPU load.

Package	EC2 Instance Type	Servers	Locks/server/sec	Total Locks/sec	CPU Usage
net/rpc	c3.2xlarge	8	(min=1884, max=2096)	15920	25%
grpc	c3.2xlarge	8	(min=798, max=923)	6884	55%

To verify that the actual (distributed) locking functionality is not influencing the tests negatively, we repeated the measurements without any server side logic (all code commented out) which gave these results (note that a single lock takes 16 messages, namely 8 locks plus 8 unlocks):

Package	EC2 Instance Type	Servers	Messages/server/sec	Total Msgs/sec	CPU Usage
net/rpc	c3.2xlarge	8	(min=43974, max=53790)	391056	30%
grpc	c3.2xlarge	8	(min=20177, max=22272)	169796	60%

Again the grpc performance is about half that of net/rpc at double the CPU load. (Note that the grpc measurements were done using a separate branch, see https://github.com/fwessels/dsync/tree/performance-grpc).

We also found another reference graph that shows "Go sync" performance: https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5760820306771968&widget=828849126&container=1808124444
This shows that the performance for “Go sync” is about 48K QPS for an 8 core server (equivalent of 6K/per core...) which seems to be inline with the results that we are getting. C++ is close to an order of magnitude faster (albeit async) which looks more like it.

So it looks like grpc adds significant overhead compared to net/rpc so we’ll hold off on migrating just yet…

Are there any plans to bring the performance of grpc up to par with net/rpc ?

[pkieltyka]

@fwessels it would be interesting to see the performance difference with grpc using the serialization code generated by https://github.com/gogo/protobuf -- I think it'll show that grpc has lots of potential to get optimized

[fwessels]

@pkieltyka I would doubt whether this would make a big difference as the arguments in the test are really simple (just a string in and bool as return value). Do you have an idea how much difference it could make?

[robmccoll]

cannot support TLS without major work

Perhaps I'm missing something, but is it not as simple as using either Accept or ServeConn with a TLS net.Listener or TLS net.Conn?

listener, _ := tls.Listen("tcp", ":1234", config)
rpc.Accept(listener)

As you said, nice API, no need for an IDL or other dependencies, and I would add simple, composable, and powerful. It's among the easiest RPC mechanisms to implement for Go, scales well, and achieves high throughput with low latency. grpc to me isn't really an alternative as it's design is focused on completely different goals. grpc provides many additional capabilities, but at the cost of the things that make net/rpc desirable.

[danmux]

I am confused by the difference between 'deprecate' and 'freeze', is there any formality about these terms. I would typically see deprecation as frozen hoping for the ability to remove at some future time.

The complaint about https is only partly valid, I agree it is annoying to find the recipe but once found does not change much - this could be added to the examples.

I agree with others that that net/rpc is lightweight fast simple and serves many scenarios well.

I have not suffered from any of what @robpike refers to as "outstanding bugs that are hard to fix," so would be happy for these to be migrated to documented known limitations, and avoid going to any great lengths to fix these bugs.

for the reasons @robmccoll mentions I think it would be wrong to encourage folk to use alternatives like GRPC in place of net/rpc and the word 'deprecate' would discourage the otherwise perfectly sensible use of this simple useful package.

[bradfitz]

I think everybody only means "freeze".

[adg]

Freeze it!

[gopherbot]

CL https://golang.org/cl/32112 mentions this issue.