Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Commit

Permalink
Merge pull request #200 from microsoft/master
Browse files Browse the repository at this point in the history 
merge master
  • Loading branch information
@SparkSnail
SparkSnail authored Aug 6, 2019
2 parents 755ac5f + 3a6d137 commit 704b50e
Show file tree
Hide file tree
Showing 137 changed files with 2,425 additions and 955 deletions.
154 changes: 93 additions & 61 deletions README_zh_CN.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@

NNI (Neural Network Intelligence) 是自动机器学习（AutoML）的工具包。 它通过多种调优的算法来搜索最好的神经网络结构和（或）超参，并支持单机、本地多机、云等不同的运行环境。

### **NNI [v0.9](https://github.com/Microsoft/nni/releases) 已发布！**
### **NNI [v0.9](https://github.com/Microsoft/nni/releases) 已发布！ &nbsp;[<img width="48" src="docs/img/release_icon.png" />](#nni-released-reminder)**

<p align="center">
<a href="#nni-v05-has-been-released"><img src="docs/img/overview.svg" /></a>
<a href="#nni-has-been-released"><img src="docs/img/overview.svg" /></a>
</p>

<table>
Expand All @@ -28,11 +28,11 @@ NNI (Neural Network Intelligence) 是自动机器学习（AutoML）的工具包
<img src="docs/img/bar.png"/>
</td>
<td>
<b>训练服务</b>
<b>训练平台</b>
<img src="docs/img/bar.png"/>
</td>
</tr>
<tr/>
</tr>
<tr valign="top">
<td>
<ul>
Expand All @@ -46,39 +46,45 @@ NNI (Neural Network Intelligence) 是自动机器学习（AutoML）的工具包
<li>Theano</li>
</ul>
</td>
<td>
<a href="docs/zh_CN/BuiltinTuner.md">Tuner（调参器）</a>
<ul>
<li><a href="docs/zh_CN/BuiltinTuner.md#TPE">TPE</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#Random">Random Search（随机搜索）</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#Anneal">Anneal（退火算法）</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#Evolution">Naive Evolution（进化算法）</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#SMAC">SMAC</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#Batch">Batch（批处理）</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#GridSearch">Grid Search（遍历搜索）</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#Hyperband">Hyperband</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#NetworkMorphism">Network Morphism</a></li>
<li><a href="examples/tuners/enas_nni/README_zh_CN.md">ENAS</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#MetisTuner">Metis Tuner</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#BOHB">BOHB</a></li>
<li><a href="docs/zh_CN/BuiltinTuner.md#GPTuner">GP Tuner</a></li>
</ul>
<a href="docs/zh_CN/BuiltinAssessor.md">Assessor（评估器）</a>
<td align="left">
<a href="docs/en_US/Tuner/BuiltinTuner.md">Tuner（调参器）</a>
<br />
<ul>
<li><a href="docs/zh_CN/BuiltinAssessor.md#Medianstop">Median Stop</a></li>
<li><a href="docs/zh_CN/BuiltinAssessor.md#Curvefitting">Curve Fitting</a></li>
</ul>
<b style="margin-left:-20px">通用 Tuner</b>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#Random">Random Search（随机搜索）</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#Evolution">Naïve Evolution（进化算法）</a></li>
<b style="margin-left:-20px">超参 Tuner</b>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#TPE">TPE</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#Anneal">Anneal（退火算法）</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#SMAC">SMAC</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#Batch">Batch（批处理）</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#GridSearch">Grid Search（遍历搜索）</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#Hyperband">Hyperband</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#MetisTuner">Metis Tuner</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#BOHB">BOHB</a></li>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#GPTuner">GP Tuner</a></li>
<b style="margin-left:-20px">网络结构 Tuner</b>
<li><a href="docs/en_US/Tuner/BuiltinTuner.md#NetworkMorphism">Network Morphism</a></li>
<li><a href="examples/tuners/enas_nni/README.md">ENAS</a></li>
</ul>
<a href="docs/en_US/Assessor/BuiltinAssessor.md">Assessor（评估器）</a>
<ul>
<li><a href="docs/en_US/Assessor/BuiltinAssessor.md#Medianstop">Median Stop（中位数终止）</a></li>
<li><a href="docs/en_US/Assessor/BuiltinAssessor.md#Curvefitting">Curve Fitting（曲线拟合）</a></li>
</ul>
</td>
<td>
<ul>
<li><a href="docs/zh_CN/LocalMode.md">本地计算机</a></li>
<li><a href="docs/zh_CN/RemoteMachineMode.md">远程计算机</a></li>
<li><a href="docs/zh_CN/PaiMode.md">OpenPAI</a></li>
<li><a href="docs/zh_CN/KubeflowMode.md">Kubeflow</a></li>
<li><a href="docs/zh_CN/FrameworkControllerMode.md">基于 Kubernetes（AKS 等等）的 FrameworkController</a></li>
<li><a href="docs/en_US/TrainingService/LocalMode.md">本机</a></li>
<li><a href="docs/en_US/TrainingService/RemoteMachineMode.md">远程计算机</a></li>
<li><b>基于 Kubernetes 的平台</b></li>
<ul><li><a href="docs/en_US/TrainingService/PaiMode.md">OpenPAI</a></li>
<li><a href="docs/en_US/TrainingService/KubeflowMode.md">Kubeflow</a></li>
<li><a href="docs/en_US/TrainingService/FrameworkControllerMode.md">基于 Kubernetes（AKS 等）的 FrameworkController</a></li>
</ul>
</ul>
</td>
</tr>
</tr>
</tbody>
</table>

Expand Down Expand Up @@ -122,7 +128,7 @@ python -m pip install --upgrade nni

* 如果需要将 NNI 安装到自己的 home 目录中，可使用 `--user`，这样也不需要任何特殊权限。
* 目前，Windows 上的 NNI 支持本机，远程和 OpenPAI 模式。 强烈推荐使用 Anaconda 或 Miniconda 在 Windows 上安装 NNI。
* 如果遇到如`Segmentation fault` 这样的任何错误请参考[常见问题](docs/zh_CN/FAQ.md)。
* 如果遇到如`Segmentation fault` 这样的任何错误请参考[常见问题](docs/zh_CN/Tutorial/FAQ.md)。

**通过源代码安装**

Expand All @@ -133,7 +139,7 @@ Linux 和 macOS
* 在 `python >= 3.5` 的环境中运行命令： `git` 和 `wget`，确保安装了这两个组件。

```bash
git clone -b v0.8 https://github.com/Microsoft/nni.git
git clone -b v0.9 https://github.com/Microsoft/nni.git
cd nni
source install.sh
```
Expand All @@ -143,14 +149,14 @@ Windows
* 在 `python >=3.5` 的环境中运行命令： `git` 和 `PowerShell`，确保安装了这两个组件。

```bash
git clone -b v0.8 https://github.com/Microsoft/nni.git
git clone -b v0.9 https://github.com/Microsoft/nni.git
cd nni
powershell -ExecutionPolicy Bypass -file install.ps1
```

参考[安装 NNI](docs/zh_CN/Installation.md) 了解系统需求。
参考[安装 NNI](docs/zh_CN/Tutorial/Installation.md) 了解系统需求。

Windows 上参考 [Windows 上使用 NNI](docs/zh_CN/NniOnWindows.md)。
Windows 上参考 [Windows 上使用 NNI](docs/zh_CN/Tutorial/NniOnWindows.md)。

**验证安装**

Expand All @@ -159,7 +165,7 @@ Windows 上参考 [Windows 上使用 NNI](docs/zh_CN/NniOnWindows.md)。
* 通过克隆源代码下载示例。

```bash
git clone -b v0.8 https://github.com/Microsoft/nni.git
git clone -b v0.9 https://github.com/Microsoft/nni.git
```

Linux 和 macOS
Expand Down Expand Up @@ -207,7 +213,7 @@ You can use these commands to get more information about the experiment
-----------------------------------------------------------------------
```

* 在浏览器中打开 `Web UI url`，可看到下图的 Experiment 详细信息，以及所有的 Trial 任务。 查看[这里](docs/zh_CN/WebUI.md)的更多页面。
* 在浏览器中打开 `Web UI url`，可看到下图的 Experiment 详细信息，以及所有的 Trial 任务。 查看[这里](docs/zh_CN/Tutorial/WebUI.md)的更多页面。

<table style="border: none">
<th><img src="./docs/img/webui_overview_page.png" alt="drawing" width="395"/></th>
Expand All @@ -216,43 +222,69 @@ You can use these commands to get more information about the experiment

## **文档**

主要文档都可以在[这里](https://nni.readthedocs.io/cn/latest/Overview.html)找到，文档均从本代码库生成。
点击阅读：

* [NNI 概述](docs/zh_CN/Overview.md)
* [快速入门](docs/zh_CN/QuickStart.md)
* [快速入门](docs/en_US/Tutorial/QuickStart.md)
* [贡献](docs/en_US/Tutorial/Contributing.md)
* [示例](docs/en_US/examples.rst)
* [参考](docs/en_US/reference.rst)
* [Web 界面教程](docs/en_US/Tutorial/WebUI.md)

## **入门**

* [安装 NNI](docs/zh_CN/Installation.md)
* [使用命令行工具 nnictl](docs/zh_CN/Nnictl.md)
* [使用 NNIBoard](docs/zh_CN/WebUI.md)
* [如何定义搜索空间](docs/zh_CN/SearchSpaceSpec.md)
* [如何编写 Trial 代码](docs/zh_CN/Trials.md)
* [如何选择 Tuner、搜索算法](docs/zh_CN/BuiltinTuner.md)
* [配置 Experiment](docs/zh_CN/ExperimentConfig.md)
* [如何使用 Annotation](docs/zh_CN/Trials.md#nni-python-annotation)
* [安装 NNI](docs/en_US/Tutorial/Installation.md)
* [使用命令行工具 nnictl](docs/en_US/Tutorial/Nnictl.md)
* [使用 NNIBoard](docs/en_US/Tutorial/WebUI.md)
* [如何定义搜索空间](docs/en_US/Tutorial/SearchSpaceSpec.md)
* [如何实现 Trial 代码](docs/en_US/TrialExample/Trials.md)
* [如何选择 Tuner、搜索算法](docs/en_US/Tuner/BuiltinTuner.md)
* [配置 Experiment](docs/en_US/Tutorial/ExperimentConfig.md)
* [如何使用 Annotation](docs/en_US/TrialExample/Trials.md#nni-python-annotation)

## **教程**

* [在本机运行 Experiment (支持多 GPU 卡)](docs/zh_CN/LocalMode.md)
* [在多机上运行 Experiment](docs/zh_CN/RemoteMachineMode.md)
* [在 OpenPAI 上运行 Experiment](docs/zh_CN/PaiMode.md)
* [在 Kubeflow 上运行 Experiment。](docs/zh_CN/KubeflowMode.md)
* [尝试不同的 Tuner](docs/zh_CN/tuners.rst)
* [尝试不同的 Assessor](docs/zh_CN/assessors.rst)
* [实现自定义 Tuner](docs/zh_CN/CustomizeTuner.md)
* [实现自定义 Assessor](docs/zh_CN/CustomizeAssessor.md)
* [使用进化算法为阅读理解任务找到好模型](examples/trials/ga_squad/README_zh_CN.md)
* [在 OpenPAI 上运行 Experiment](docs/en_US/TrainingService/PaiMode.md)
* [在 Kubeflow 上运行 Experiment](docs/en_US/TrainingService/KubeflowMode.md)
* [在本机运行 Experiment (支持多 GPU 卡)](docs/en_US/TrainingService/LocalMode.md)
* [在多机上运行 Experiment](docs/en_US/TrainingService/RemoteMachineMode.md)
* [尝试不同的 Tuner](docs/en_US/Tuner/BuiltinTuner.md)
* [尝试不同的 Assessor](docs/en_US/Assessor/BuiltinAssessor.md)
* [实现自定义 Tuner](docs/en_US/Tuner/CustomizeTuner.md)
* [实现自定义 Assessor](docs/en_US/Assessor/CustomizeAssessor.md)
* [使用进化算法为阅读理解任务找到好模型](docs/en_US/TrialExample/SquadEvolutionExamples.md)

## **贡献**

欢迎贡献代码或提交建议，可在 [GitHub issues](https://github.com/Microsoft/nni/issues) 跟踪需求和 Bug。
非常欢迎通过各种方式参与此项目，例如：

* 审查[源代码改动](https://github.com/microsoft/nni/pulls)
* 审查[文档](https://github.com/microsoft/nni/tree/master/docs)中从拼写错误到新内容的任何内容，并提交拉取请求。
* 找到标有 ['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) 或 ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) 标签的 Issue。这些都是简单的 Issue，新的贡献者可以从这些问题开始。

在提交代码前，需要遵循以下的简单准则：

* [如何调试](docs/en_US/Tutorial/HowToDebug.md)
* [代码风格和命名约定](docs/en_US/Tutorial/Contributing.md)
* 如何设置 [NNI 开发环境](docs/zh_CN/Tutorial/SetupNniDeveloperEnvironment.md)
* 查看[贡献说明](docs/en_US/Tutorial/Contributing.md)并熟悉 NNI 的代码贡献指南

## **外部代码库**

推荐新贡献者从标有 **good first issue** 的简单需求开始。
下面是一些贡献者为 NNI 提供的使用示例 谢谢可爱的贡献者！ 欢迎越来越多的人加入我们！

如要安装 NNI 开发环境，参考：[配置 NNI 开发环境](docs/zh_CN/SetupNniDeveloperEnvironment.md)。
* 在 NNI 中运行 [ENAS](examples/tuners/enas_nni/README_zh_CN.md)
* 在 NNI 中运行 [神经网络架构结构搜索](examples/trials/nas_cifar10/README_zh_CN.md)

在写代码之前，请查看并熟悉 NNI 代码贡献指南：[贡献](docs/zh_CN/Contributing.md)。
## **反馈**

我们正在编写[如何调试](docs/zh_CN/HowToDebug.md) 的页面，欢迎提交建议和问题。
* [报告 Bug](https://github.com/microsoft/nni/issues/new/choose)。

* [请求新功能](https://github.com/microsoft/nni/issues/new/choose).
* 在 [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) 中参与讨论
* 在 [Stack Overflow](https://stackoverflow.com/questions/tagged/nni?sort=Newest&edited=true) 上使用 nni 的标签提问，或[在 Github 上提交 Issue](https://github.com/microsoft/nni/issues/new/choose)。
* 我们正在实现[如何调试](docs/zh_CN/Tutorial/HowToDebug.md)的页面，欢迎提交建议和问题。

## **许可协议**

Expand Down
2 changes: 1 addition & 1 deletion azure-pipelines.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
displayName: 'Install nni toolkit via source code'
- script: |
python3 -m pip install flake8 --user
IGNORE=./tools/nni_annotation/testcase/*:F821,./examples/trials/mnist-nas/mnist.py:F821
IGNORE=./tools/nni_annotation/testcase/*:F821,./examples/trials/mnist-nas/*/mnist*.py:F821
python3 -m flake8 . --count --per-file-ignores=$IGNORE --select=E9,F63,F72,F82 --show-source --statistics
displayName: 'Run flake8 tests to find Python syntax errors and undefined names'
- script: |
Expand Down
12 changes: 12 additions & 0 deletions crowdin.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
files:
- source: '/**/*.[mM][dD]'
ignore:
- '*_%locale_with_underscore%.md'
- /docs
- /%locale_with_underscore%
- '**/ISSUE_TEMPLATE/**'
translation: /%original_path%/%file_name%_%locale_with_underscore%.md
- source: /docs/en_US/**/*
ignore:
- /docs/%locale_with_underscore%/**/*.*
translation: /docs/%locale_with_underscore%/**/%original_file_name%
2 changes: 1 addition & 1 deletion docs/en_US/AdvancedFeature/MultiPhase.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ trial:

### Write a tuner that leverages multi-phase:

Before writing a multi-phase tuner, we highly suggest you to go through [Customize Tuner](https://nni.readthedocs.io/en/latest/Customize_Tuner.html). Same as writing a normal tuner, your tuner needs to inherit from `Tuner` class. When you enable multi-phase through configuration (set `multiPhase` to true), your tuner will get an additional parameter `trial_job_id` via tuner's following methods:
Before writing a multi-phase tuner, we highly suggest you to go through [Customize Tuner](https://nni.readthedocs.io/en/latest/Tuner/CustomizeTuner.html). Same as writing a normal tuner, your tuner needs to inherit from `Tuner` class. When you enable multi-phase through configuration (set `multiPhase` to true), your tuner will get an additional parameter `trial_job_id` via tuner's following methods:
```
generate_parameters
generate_multiple_parameters
Expand Down
8 changes: 4 additions & 4 deletions docs/en_US/CommunitySharings/ParallelizingTpeSearch.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

TPE approaches were actually run asynchronously in order to make use of multiple compute nodes and to avoid wasting time waiting for trial evaluations to complete. For the TPE approach, the so-called constant liar approach was used: each time a candidate point x∗ was proposed, a fake fitness evaluation of the y was assigned temporarily, until the evaluation completed and reported the actual loss f(x∗).

## Introducion and Problems
## Introduction and Problems

### Sequential Model-based Global Optimization

Expand All @@ -19,7 +19,7 @@ Since calculation of p(y|x) is expensive, TPE approach modeled p(y|x) by p(x|y)
![](../../img/parallel_tpe_search_tpe.PNG)

where l(x) is the density formed by using the observations {x(i)} such that corresponding loss
f(x(i)) was less than y∗ and g(x) is the density formed by using the remaining observations. TPE algorithm depends on a y∗ that is larger than the best observed f(x) so that some points can be used to form l(x). The TPE algorithm chooses y∗ to be some quantile γ of the observed y values, so that p(y<`y∗`) = γ, but no specific model for p(y) is necessary. The tree-structured form of l and g makes it easy to draw manycandidates according to l and evaluate them according to g(x)/l(x). On each iteration, the algorithm returns the candidate x∗ with the greatest EI.
f(x(i)) was less than y∗ and g(x) is the density formed by using the remaining observations. TPE algorithm depends on a y∗ that is larger than the best observed f(x) so that some points can be used to form l(x). The TPE algorithm chooses y∗ to be some quantile γ of the observed y values, so that p(y<`y∗`) = γ, but no specific model for p(y) is necessary. The tree-structured form of l and g makes it easy to draw many candidates according to l and evaluate them according to g(x)/l(x). On each iteration, the algorithm returns the candidate x∗ with the greatest EI.

Here is a simulation of the TPE algorithm in a two-dimensional search space. The difference of background color represents different values. It can be seen that TPE combines exploration and exploitation very well. (Black indicates the points of this round samples, and yellow indicates the points has been taken in the history.)

Expand Down Expand Up @@ -69,13 +69,13 @@ We have simulated the method above. The following figure shows the result of usi

### Branin-Hoo

The four optimization strtigeies presented in the last section are now complared on the Branin-Hoo function which is a classical test-case in global optimization.
The four optimization strategies presented in the last section are now compared on the Branin-Hoo function which is a classical test-case in global optimization.

![](../../img/parallel_tpe_search_branin.PNG)

The recommended values of a, b, c, r, s and t are: a = 1, b = 5.1 ⁄ (4π2), c = 5 ⁄ π, r = 6, s = 10 and t = 1 ⁄ (8π). This function has three global minimizers(-3.14, 12.27), (3.14, 2.27), (9.42, 2.47).

Next is the comparaison of the q-EI associated with the q first points (q ∈ [1,10]) given by the constant liar strategies (min and max), 2000 q-points designs uniformly drawn for every q, and 2000 q-points LHS designs taken at random for every q.
Next is the comparison of the q-EI associated with the q first points (q ∈ [1,10]) given by the constant liar strategies (min and max), 2000 q-points designs uniformly drawn for every q, and 2000 q-points LHS designs taken at random for every q.

![](../../img/parallel_tpe_search_result.PNG)

Expand Down
Loading

0 comments on commit 704b50e

Please sign in to comment.