Update librdkafka to 1.0.0+

[yongtang]

This is an issue to update librdkafka to 1.0.0+. The biggest motivation for update is that our current version of librdkafka does not work correctly with OS X 10.13, due to Blizzard/node-rdkafka#686 (comment)

Since we are publishing OS X 10.13 compatible wheels now, it is imperative to update librdkafka so that tensorflow-io on 10.13 works correctly.

Currently, there is another issue with librdkafka that is causing segmentation fault if we update, it is being tracked in confluentinc/librdkafka#2716

[jjedele]

Ahrg, just downloaded 8GB Xcode to figure out it has this issue 😓

[jjedele]

Documenting this here in case anyone else runs into the problem.

This is what the error will actually look like when you run into it:

In [2]: tf.load_op_library("libtensorflow_io.so")
---------------------------------------------------------------------------
NotFoundError                             Traceback (most recent call last)
<ipython-input-2-cea886c59c24> in <module>
----> 1 tf.load_op_library("libtensorflow_io.so")

~/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/load_library.py in load_op_library(library_filename)
     55     RuntimeError: when unable to load the library or get the python wrappers.
     56   """
---> 57   lib_handle = py_tf.TF_LoadLibrary(library_filename)
     58   try:
     59     wrappers = py_tf.GetPythonWrappers(py_tf.TF_GetOpList(lib_handle))

NotFoundError: dlopen(libtensorflow_io.so, 6): Symbol not found: _timespec_get
  Referenced from: libtensorflow_io.so (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libSystem.B.dylib
 in libtensorflow_io.so

The problem seems to be the Xcode version rather than the OS version. I'm running macOS 10.4.6. With Xcode 11.x I got above error. Downgrading to Xcode 10.3 seems to have fixed the problem for me. Old versions can be downloaded here: https://developer.apple.com/download/.

I had to following steps after downgrading:

sudo xcode-select -s /Applications/Xcode.app/Contents/Developer # make sure right version is selected
sudo xcodebuild -license accept # accept license
bazel clean --expunge # clean bazel cache completely

# then build again
bazel build -s --verbose_failures //tensorflow_io/...

[yongtang]

@jjedele Sorry for the inconvenience. In order to move forward on confluentinc/librdkafka#2716 will need a simplified crash example. I will give another try today.

[yongtang]

I am not able to find out the reason yet, though if I comment out the line then librdkafka 1.4.0 will work without crash.
https://github.com/edenhill/librdkafka/blob/master/src/rdkafka_cgrp.c#L2727-L2733

Will play a little more, and update the librdkafka either way.

[yongtang]

@jjedele Added a PR #838 to upgrade librdkafka to 1.4.0-RC2.

[yongtang]

@jjedele The librdkafka has been upgraded to 1.4.0RC2 in #838.