TF Dynamic kernel design.

[gunan]

Please leave comments to this PR. Comment period will be open until 2018-06-19

cc @ewilderj @martinwicke

Dynamic Loading of Kernels in TensorFlow

Status	Proposed
Author(s)	Gunhan Gulsoy (Google)
Sponsor	Martin Wicke (Google)
Updated	2018-06-04

Objective

This document describes a new way to create and deploy new kernels for
TensorFlow. We propose deploying kernels in separate shared libraries (dso,
dylib or dll) and loading these at runtime. While at the moment the scope of
this document only covers TensorFlow Python distribution, we aim to
generalize this approach for all TF distributions. With this mechanism, we
would like to create the following capabilities:

• Loading kernels dynamically at runtime from shared libraries.
• Being able to load multiple kernels for the same op/device pair, and pick the
  best one in terms of hardware compatibility and performance.
• Check the hardware and load the compatible kernels.
• Check compiler options used and load the compatible kernels.

[ewilderj]

@gunan if this passed Google internal design review, then we can merge it into rfcs/ - what's the current status of the design?

[martinwicke]

Yeah, should be merged, not closed, I think.

[ewilderj]

@gunan can you reopen this?

[gunan]

Ah, sorry looks like an update in my copy of the repository caused this. Did not realize I closed this.
I will try to see how I can reopen this.