Update docs about nested search space (#1115)

* update readme in ga_squad * update readme * fix typo * Update README.md * Update README.md * Update README.md * update readme * update * fix path * update reference * fix bug in config file * update nni_arch_overview.png * update * update * update * update home page * update default value of metis tuner * fix broken link in CommunitySharings * update docs about nested search space * update docs * rename cascding to nested * fix broken link * update * update issue link * fix typo
microsoft · May 28, 2019 · 0feed01 · 0feed01
1 parent cc961de
commit 0feed01
Show file tree

Hide file tree

Showing 10 changed files with 22 additions and 12 deletions.
diff --git a/docs/en_US/MnistExamples.md b/docs/en_US/MnistExamples.md
@@ -51,7 +51,7 @@ This example is to show how to use hyperband to tune the model. There is one mor
 
 This example is to show that NNI also support nested search space. The search space file is an example of how to define nested search space.
 
-`code directory: examples/trials/mnist-cascading-search-space/`
+`code directory: examples/trials/mnist-nested-search-space/`
 
 <a name="mnist-kubeflow-tf"></a>
 **distributed MNIST (tensorflow) using kubeflow**

diff --git a/docs/en_US/SearchSpaceSpec.md b/docs/en_US/SearchSpaceSpec.md
@@ -29,7 +29,7 @@ All types of sampling strategies and their parameter are listed here:
 
   * Which means the variable's value is one of the options. Here 'options' should be a list. Each element of options is a number of string. It could also be a nested sub-search-space, this sub-search-space takes effect only when the corresponding element is chosen. The variables in this sub-search-space could be seen as conditional variables.
 
-  * An simple [example](../../examples/trials/mnist-cascading-search-space/search_space.json) of [nested] search space definition. If an element in the options list is a dict, it is a sub-search-space, and for our built-in tuners you have to add a key '_name' in this dict, which helps you to identify which element is chosen. Accordingly, here is a [sample](../../examples/trials/mnist-cascading-search-space/sample.json) which users can get from nni with nested search space definition. Tuners which support nested search space is as follows:
+  * An simple [example](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nested-search-space/search_space.json) of [nested] search space definition. If an element in the options list is a dict, it is a sub-search-space, and for our built-in tuners you have to add a key '_name' in this dict, which helps you to identify which element is chosen. Accordingly, here is a [sample](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nested-search-space/sample.json) which users can get from nni with nested search space definition. Tuners which support nested search space is as follows:
 
     - Random Search 
     - TPE
@@ -86,9 +86,19 @@ All types of sampling strategies and their parameter are listed here:
 | Hyperband Advisor | &#10003; | &#10003; | &#10003; | &#10003;  | &#10003;    | &#10003;     | &#10003; | &#10003; | &#10003;   | &#10003;    |
 | Metis Tuner   | &#10003; | &#10003; | &#10003; | &#10003;  |     |      |  |  |    |     |
 
-Note that In Grid Search Tuner, for users' convenience, the definition of `quniform` and `qloguniform` change, where q here specifies the number of values that will be sampled. Details about them are listed as follows
 
-* Type 'quniform' will receive three values [low, high, q], where [low, high] specifies a range and 'q' specifies the number of values that will be sampled evenly. Note that q should be at least 2. It will be sampled in a way that the first sampled value is 'low', and each of the following values is (high-low)/q larger that the value in front of it.
-* Type 'qloguniform' behaves like 'quniform' except that it will first change the range to [log(low), log(high)] and sample and then change the sampled value back.
+Known Limitations:
 
-Note that Metis Tuner only support numerical `choice` now
+* Note that In Grid Search Tuner, for users' convenience, the definition of `quniform` and `qloguniform` change, where q here specifies the number of values that will be sampled. Details about them are listed as follows
+
+    * Type 'quniform' will receive three values [low, high, q], where [low, high] specifies a range and 'q' specifies the number of values that will be sampled evenly. Note that q should be at least 2. It will be sampled in a way that the first sampled value is 'low', and each of the following values is (high-low)/q larger that the value in front of it.
+
+    * Type 'qloguniform' behaves like 'quniform' except that it will first change the range to [log(low), log(high)] and sample and then change the sampled value back.
+
+* Note that Metis Tuner only supports numerical `choice` now
+
+* Note that for nested search space:
+
+    * Only Random Search/TPE/Anneal/Evolution tuner supports nested search space
+
+    * We do not support nested search space "Hyper Parameter" parallel graph now, the enhancement is being considered in #1110(https://github.com/microsoft/nni/issues/1110), any suggestions or discussions or contributions are warmly welcomed
diff --git a/docs/zh_CN/MnistExamples.md b/docs/zh_CN/MnistExamples.md
@@ -51,7 +51,7 @@
 
 此样例演示了 NNI 如何支持嵌套的搜索空间。 搜索空间文件示了如何定义嵌套的搜索空间。
 
-`代码目录: examples/trials/mnist-cascading-search-space/`
+`代码目录: examples/trials/mnist-nested-search-space/`
 
 <a name="mnist-kubeflow-tf"></a>
 **用 Kubeflow 运行分布式的 MNIST (tensorflow)**

diff --git a/docs/zh_CN/SearchSpaceSpec.md b/docs/zh_CN/SearchSpaceSpec.md
@@ -29,7 +29,7 @@
 
   * 表示变量的值是选项之一。 这里的 'options' 是一个数组。 选项的每个元素都是字符串。 也可以是嵌套的子搜索空间。此子搜索空间仅在相应的元素选中后才起作用。 该子搜索空间中的变量可看作是条件变量。
 
-  * 这是个简单的 [nested] 搜索空间定义的[示例](../../examples/trials/mnist-cascading-search-space/search_space.json)。 如果选项列表中的元素是 dict，则它是一个子搜索空间，对于内置的 Tuner，必须在此 dict 中添加键 “_name”，这有助于标识选中的元素。 相应的，这是从 NNI 中获得的嵌套搜索空间定义的[示例](../../examples/trials/mnist-cascading-search-space/sample.json)。 以下 Tuner 支持嵌套搜索空间：
+  * 这是个简单的 [nested] 搜索空间定义的[示例](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nested-search-space/search_space.json)。 如果选项列表中的元素是 dict，则它是一个子搜索空间，对于内置的 Tuner，必须在此 dict 中添加键 “_name”，这有助于标识选中的元素。 相应的，这是从 NNI 中获得的嵌套搜索空间定义的[示例](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nested-search-space/sample.json)。 以下 Tuner 支持嵌套搜索空间：
 
     * Random Search（随机搜索） 
     * TPE

diff --git a/...s/mnist-cascading-search-space/config.yml → ...ials/mnist-nested-search-space/config.yml b/...s/mnist-cascading-search-space/config.yml → ...ials/mnist-nested-search-space/config.yml
@@ -1,5 +1,5 @@
 authorName: default
-experimentName: mnist-cascading-search-space
+experimentName: mnist-nested-search-space
 trialConcurrency: 2
 maxExecDuration: 1h
 maxTrialNum: 100

diff --git a/...als/mnist-cascading-search-space/mnist.py → ...trials/mnist-nested-search-space/mnist.py b/...als/mnist-cascading-search-space/mnist.py → ...trials/mnist-nested-search-space/mnist.py
@@ -14,7 +14,7 @@
 
 import nni
 
-logger = logging.getLogger('mnist_cascading_search_space')
+logger = logging.getLogger('mnist_nested_search_space')
 FLAGS = None
 
 class MnistNetwork(object):

diff --git a/...st-cascading-search-space/requirments.txt → ...mnist-nested-search-space/requirments.txt b/...st-cascading-search-space/requirments.txt → ...mnist-nested-search-space/requirments.txt
diff --git a/.../mnist-cascading-search-space/sample.json → ...als/mnist-nested-search-space/sample.json b/.../mnist-cascading-search-space/sample.json → ...als/mnist-nested-search-space/sample.json
diff --git a/...-cascading-search-space/search_space.json → ...ist-nested-search-space/search_space.json b/...-cascading-search-space/search_space.json → ...ist-nested-search-space/search_space.json
diff --git a/...les/mnist-cascading-search-space.test.yml → ...amples/mnist-nested-search-space.test.yml b/...les/mnist-cascading-search-space.test.yml → ...amples/mnist-nested-search-space.test.yml
@@ -3,7 +3,7 @@ experimentName: default_test
 maxExecDuration: 5m
 maxTrialNum: 4
 trialConcurrency: 2
-searchSpacePath: ../../../examples/trials/mnist-cascading-search-space/search_space.json
+searchSpacePath: ../../../examples/trials/mnist-nested-search-space/search_space.json
 
 tuner:
   #choice: TPE, Random, Anneal, Evolution
@@ -13,7 +13,7 @@ assessor:
   classArgs:
     optimize_mode: maximize
 trial:
-  codeDir: ../../../examples/trials/mnist-cascading-search-space
+  codeDir: ../../../examples/trials/mnist-nested-search-space
   command: python3 mnist.py --batch_num 100
   gpuNum: 0