[RFC] Custom subgraph property enhancements

[samskalicky]

Description

Request for comments on the next PR for enhancing custom subgraph property support

Heres some suggestions from the initial PR (Part 1):

• docs, readme, tutorial Dynamic subgraph property doc #17585
• dynamic graph passes
• move builds of so's for customOp/subgraphProp from top level directory
• Load libraries that are found as siblings next to libmxnet.so/dll. This will allow us to package external libraries into the pip wheel (ie. MKLDNN, TensorRT)
• Load libraries that are specified in an environment variable like MXNET_EXT_LIBS=/path/to/libmylib.so
• Apply subgraphBackend during bind when specified by an environment variable like MXNET_SUBGRAPH_BACKEND=myProp

Suggestions from the compile PR:

• Enable removing params that are fully consumed by a subgraph during compilation to avoid duplication of tensor values

References

• initial PR (Part 1): Dynamic subgraph property #17034
• compile PR Dynamic subgraph compile support #17623

[samskalicky]

@mxnet-label-bot add [RFC]

[samskalicky]

Current implementation (as of 3/18/2020) only supports combining ops into subgraphs generically. does not allow for fine grained control to say which ops should go into which subgraphs. Need to modify supportedOps API to change the ids vector from bool (supported or not) to integer (to say which subgraph this node should go in to). This will allow for coloring and fine grained control of ops in a subgraph.
https://github.com/apache/incubator-mxnet/blob/07d0b738384cba011ce519bcc41aeeaaafcd22c1/example/extensions/lib_subgraph/subgraph_lib.cc#L179-L181
Will also need to modify custom_subgraph_property.h selector to use these int values to define the subgraph instead of just looking for existence of the operator name.
https://github.com/apache/incubator-mxnet/blob/41d534be3ffd7b45846f555366230f4e811d8751/src/operator/subgraph/partitioner/custom_subgraph_property.h#L47-L54

Open question:
How do we handle the cases where we want to remove args/aux from the top level of the model, migrating those into the subgraph? If we compile a subgraph with Args/Aux then those tensor values are effectively frozen in the compiled binary so we need to be clear with users that they can no longer be changed in the top level model

[samskalicky]

Need to support removing args/aux from custom graph pass, re-using the optimized symbol when calling optimize_for in Gluon repeatedly