Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

merge from master #2019

Merged
merged 24 commits into from
Feb 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
720e004
refactor the index of readthedocs
QuanluZhang Jan 10, 2020
17dd7fc
Merge branch 'dev-refactor-doc' of https://github.com/Microsoft/nni i…
QuanluZhang Jan 19, 2020
8d2b863
Add Docs for NNI AutoFeatureEng (#1976)
JSong-Jia Jan 22, 2020
85d5ecb
Update NNI_AutoFeatureEng.md (#1979)
scarlett2018 Jan 22, 2020
71fbff1
Refine document for installation and platform supports (#1978)
squirrelsc Jan 23, 2020
049634f
Chinese translation (#1936)
squirrelsc Jan 23, 2020
5dde0d8
support create ssh connection by using sshkey (issue #1950) (#1957)
tyusr Feb 3, 2020
c55c5f4
fix support setting nniManagerIp in PAI (#1987)
SparkSnail Feb 3, 2020
1c54b40
Change validation order in machineList (#1966)
SparkSnail Feb 4, 2020
8e953fc
Typo (#1992)
Crissman Feb 4, 2020
26aa113
Merge pai config (#1965)
SparkSnail Feb 7, 2020
d7920fd
Add foreground mode in nnictl (#1956)
SparkSnail Feb 7, 2020
7425098
Fix issue #2005 (https://github.com/microsoft/nni/issues/2005) (#2012)
RizhaoCai Feb 7, 2020
b49b38f
Add unit tests for NAS (#1954)
ultmaster Feb 7, 2020
649a9c3
Add frameworkcontroller pipeline (#1971)
SparkSnail Feb 7, 2020
e8d253c
Fix pai examples (#1996)
SparkSnail Feb 7, 2020
3c833f3
Refine doc and fix bug for testing MedianstopAssessor (#1998)
Sundrops Feb 7, 2020
0167180
Merge branch 'dev-refactor-doc' of https://github.com/Microsoft/nni i…
QuanluZhang Feb 8, 2020
fdfff50
Add PAI environment setup steps in DOC (#2011)
SparkSnail Feb 8, 2020
6932430
update index
QuanluZhang Feb 8, 2020
abac245
update
QuanluZhang Feb 8, 2020
bc0fd66
update
QuanluZhang Feb 9, 2020
eaf803f
Merge branch 'master' of https://github.com/Microsoft/nni into doc-up…
QuanluZhang Feb 9, 2020
7a76780
update
QuanluZhang Feb 9, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 26 additions & 53 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
</ul>
</ul>
</td>
</tr>
</tr>
<tr align="center" valign="bottom">
</td>
</tr>
Expand All @@ -193,18 +193,18 @@ Within the following table, we summarized the current NNI capabilities, we are g
<li><a href="docs/en_US/TrainingService/SupportTrainingService.md">Support TrainingService</li>
<li><a href="docs/en_US/TrainingService/HowToImplementTrainingService.md">Implement TrainingService</a></li>
</ul>
</td>
</tr>
</td>
</tr>
</tbody>
</table>

## **Install & Verify**
## **Installation**

**Install through pip**
### **Install**

* We support Linux, MacOS and Windows (local, remote and pai mode) in current stage, Ubuntu 16.04 or higher, MacOS 10.14.1 along with Windows 10.1809 are tested and supported. Simply run the following `pip install` in an environment that has `python >= 3.5`.
NNI supports and is tested on Ubuntu >= 16.04, macOS >= 10.14.1, and Windows 10 >= 1809. Simply run the following `pip install` in an environment that has `python 64-bit >= 3.5`.

Linux and MacOS
Linux or macOS

```bash
python3 -m pip install --upgrade nni
Expand All @@ -216,65 +216,39 @@ Windows
python -m pip install --upgrade nni
```

Note:

* `--user` can be added if you want to install NNI in your home directory, which does not require any special privileges.
* Currently NNI on Windows support local, remote and pai mode. Anaconda or Miniconda is highly recommended to install NNI on Windows.
* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.md)

**Install through source code**

* We support Linux (Ubuntu 16.04 or higher), MacOS (10.14.1) and Windows (10.1809) in our current stage.

Linux and MacOS

* Run the following commands in an environment that has `python >= 3.5`, `git` and `wget`.

```bash
git clone -b v1.3 https://github.com/Microsoft/nni.git
cd nni
source install.sh
```

Windows

* Run the following commands in an environment that has `python >=3.5`, `git` and `PowerShell`
If you want to try latest code, please [install NNI](docs/en_US/Tutorial/Installation.md) from source code.

```bash
git clone -b v1.3 https://github.com/Microsoft/nni.git
cd nni
powershell -ExecutionPolicy Bypass -file install.ps1
```
For detail system requirements of NNI, please refer to [here](docs/en_US/Tutorial/Installation.md#system-requirements).

For the system requirements of NNI, please refer to [Install NNI](docs/en_US/Tutorial/Installation.md)
Note:

For NNI on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/NniOnWindows.md)
* If there is any privilege issue, add `--user` to install NNI in the user directory.
* Currently NNI on Windows supports local, remote and pai mode. Anaconda or Miniconda is highly recommended to install NNI on Windows.
* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.md). For FAQ on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/NniOnWindows.md).

**Verify install**
### **Verify installation**

The following example is an experiment built on TensorFlow. Make sure you have **TensorFlow 1.x installed** before running it. Note that **currently Tensorflow 2.0 is NOT supported**.
The following example is built on TensorFlow 1.x. Make sure **TensorFlow 1.x is used** when running it.

* Download the examples via clone the source code.

```bash
git clone -b v1.3 https://github.com/Microsoft/nni.git
```

Linux and MacOS
```bash
git clone -b v1.3 https://github.com/Microsoft/nni.git
```

* Run the MNIST example.

```bash
nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
```
Linux or macOS

Windows
```bash
nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
```

* Run the MNIST example.
Windows

```bash
nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
```
```bash
nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
```

* Wait for the message `INFO: Successfully started experiment!` in the command line. This message indicates that your experiment has been successfully started. You can explore the experiment using the `Web UI url`.

Expand Down Expand Up @@ -371,4 +345,3 @@ We encourage researchers and students leverage these projects to accelerate the
## **License**

The entire codebase is under [MIT license](LICENSE)

99 changes: 38 additions & 61 deletions README_zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

* * *

[![MIT 许可证](https://img.shields.io/badge/license-MIT-brightgreen.svg)](LICENSE) [![生成状态](https://msrasrg.visualstudio.com/NNIOpenSource/_apis/build/status/Microsoft.nni)](https://msrasrg.visualstudio.com/NNIOpenSource/_build/latest?definitionId=6) [![问题](https://img.shields.io/github/issues-raw/Microsoft/nni.svg)](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen) [![Bug](https://img.shields.io/github/issues/Microsoft/nni/bug.svg)](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3Abug) [![拉取请求](https://img.shields.io/github/issues-pr-raw/Microsoft/nni.svg)](https://github.com/Microsoft/nni/pulls?q=is%3Apr+is%3Aopen) [![版本](https://img.shields.io/github/release/Microsoft/nni.svg)](https://github.com/Microsoft/nni/releases) [![进入 https://gitter.im/Microsoft/nni 聊天室提问](https://badges.gitter.im/Microsoft/nni.svg)](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![文档状态](https://readthedocs.org/projects/nni/badge/?version=latest)](https://nni.readthedocs.io/zh/latest/?badge=latest)
[![MIT 许可证](https://img.shields.io/badge/license-MIT-brightgreen.svg)](LICENSE) [![生成状态](https://msrasrg.visualstudio.com/NNIOpenSource/_apis/build/status/integration-test-local?branchName=master)](https://msrasrg.visualstudio.com/NNIOpenSource/_build/latest?definitionId=17&branchName=master) [![问题](https://img.shields.io/github/issues-raw/Microsoft/nni.svg)](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen) [![Bug](https://img.shields.io/github/issues/Microsoft/nni/bug.svg)](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3Abug) [![拉取请求](https://img.shields.io/github/issues-pr-raw/Microsoft/nni.svg)](https://github.com/Microsoft/nni/pulls?q=is%3Apr+is%3Aopen) [![版本](https://img.shields.io/github/release/Microsoft/nni.svg)](https://github.com/Microsoft/nni/releases) [![进入 https://gitter.im/Microsoft/nni 聊天室提问](https://badges.gitter.im/Microsoft/nni.svg)](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![文档状态](https://readthedocs.org/projects/nni/badge/?version=latest)](https://nni.readthedocs.io/zh/latest/?badge=latest)

[English](README.md)

Expand Down Expand Up @@ -83,6 +83,7 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。
<li><a href="docs/zh_CN/TrialExample/GbdtExample.md">Auto-gbdt</a></li>
<li><a href="docs/zh_CN/TrialExample/Cifar10Examples.md">Cifar10-pytorch</li></a>
<li><a href="docs/zh_CN/TrialExample/SklearnExamples.md">Scikit-learn</a></li>
<li><a href="docs/zh_CN/TrialExample/EfficientNet.md">EfficientNet</a></li>
<a href="docs/zh_CN/SupportedFramework_Library.md">更多...</a><br/>
</ul>
</ul>
Expand Down Expand Up @@ -121,6 +122,7 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。
<li><a href="docs/zh_CN/NAS/Overview.md#enas">ENAS</a></li>
<li><a href="docs/zh_CN/NAS/Overview.md#darts">DARTS</a></li>
<li><a href="docs/zh_CN/NAS/Overview.md#p-darts">P-DARTS</a></li>
<li><a href="docs/zh_CN/NAS/Overview.md#cdarts">CDARTS</a></li>
<li><a href="docs/zh_CN/Tuner/BuiltinTuner.md#NetworkMorphism">Network Morphism</a> </li>
</ul>
</ul>
Expand Down Expand Up @@ -160,7 +162,7 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。
</ul>
</ul>
</td>
</tr>
</tr>
<tr align="center" valign="bottom">
</td>
</tr>
Expand All @@ -186,18 +188,18 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。
<li><a href="docs/zh_CN/TrainingService/SupportTrainingService.md">支持训练平台</li>
<li><a href="docs/zh_CN/TrainingService/HowToImplementTrainingService.md">实现训练平台</a></li>
</ul>
</td>
</tr>
</td>
</tr>
</tbody>
</table>

## **安装和验证**
## **安装**

**通过 pip 命令安装**
### **安装**

* 当前支持 Linux,MacOS 和 Windows(本机,远程,OpenPAI 模式),在 Ubuntu 16.04 或更高版本,MacOS 10.14.1 以及 Windows 10.1809 上进行了测试。 在 `python >= 3.5` 的环境中,只需要运行 `pip install` 即可完成安装。
NNI 支持并在 Ubuntu >= 16.04, macOS >= 10.14.1, 和 Windows 10 >= 1809 通过了测试。 在 `python 64-bit >= 3.5` 的环境中,只需要运行 `pip install` 即可完成安装。

Linux macOS
Linux macOS

```bash
python3 -m pip install --upgrade nni
Expand All @@ -209,65 +211,39 @@ Windows
python -m pip install --upgrade nni
```

注意:

* 如果需要将 NNI 安装到自己的 home 目录中,可使用 `--user`,这样也不需要任何特殊权限。
* 目前,Windows 上的 NNI 支持本机,远程和 OpenPAI 模式。 强烈推荐使用 Anaconda 或 Miniconda 在 Windows 上安装 NNI。
* 如果遇到如`Segmentation fault` 这样的任何错误请参考[常见问题](docs/zh_CN/Tutorial/FAQ.md)。

**通过源代码安装**

* 当前支持 Linux(Ubuntu 16.04 或更高版本),MacOS(10.14.1)以及 Windows 10(1809 版)。

Linux 和 MacOS

* 在 `python >= 3.5` 的环境中运行命令: `git` 和 `wget`,确保安装了这两个组件。

```bash
git clone -b v1.3 https://github.com/Microsoft/nni.git
cd nni
source install.sh
```

Windows

* 在 `python >=3.5` 的环境中运行命令: `git` 和 `PowerShell`,确保安装了这两个组件。
如果想要尝试最新代码,可通过源代码[安装 NNI](docs/zh_CN/Tutorial/Installation.md)。

```bash
git clone -b v1.3 https://github.com/Microsoft/nni.git
cd nni
powershell -ExecutionPolicy Bypass -file install.ps1
```
有关 NNI 的详细系统要求,参考[这里](docs/zh_CN/Tutorial/Installation.md#system-requirements)。

参考[安装 NNI](docs/zh_CN/Tutorial/Installation.md) 了解系统需求。
注意:

Windows 上参考 [Windows 上使用 NNI](docs/zh_CN/Tutorial/NniOnWindows.md)。
* 如果遇到任何权限问题,可添加 `--user` 在用户目录中安装 NNI。
* 目前,Windows 上的 NNI 支持本机,远程和 OpenPAI 模式。 强烈推荐使用 Anaconda 或 Miniconda 在 Windows 上安装 NNI。
* 如果遇到如 `Segmentation fault` 等错误参考[常见问题](docs/zh_CN/Tutorial/FAQ.md)。 Windows 上的 FAQ 参考[在 Windows 上使用 NNI](docs/zh_CN/Tutorial/NniOnWindows.md)。

**验证安装**
### **验证安装**

以下示例 Experiment 依赖于 TensorFlow 。 在运行前确保安装了 **TensorFlow 1.x**。 注意,**目前不支持 TensorFlow 2.0**。
以下示例基于 TensorFlow 1.x 。确保运行环境中使用的的是 ** TensorFlow 1.x**。

* 通过克隆源代码下载示例。

```bash
git clone -b v1.3 https://github.com/Microsoft/nni.git
```

Linux 和 MacOS

* 运行 MNIST 示例。

```bash
nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
```

Windows

```bash
git clone -b v1.3 https://github.com/Microsoft/nni.git
```

* 运行 MNIST 示例。

```bash
nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
```

Linux 或 macOS

```bash
nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
```

Windows

```bash
nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
```

* 在命令行中等待输出 `INFO: Successfully started experiment!`。 此消息表明 Experiment 已成功启动。 通过命令行输出的 `Web UI url` 来访问 Experiment 的界面。

Expand Down Expand Up @@ -319,11 +295,12 @@ You can use these commands to get more information about the experiment

该项目采用了 [ Microsoft 开源行为准则 ](https://opensource.microsoft.com/codeofconduct/)。 有关详细信息,请参阅[常见问题解答](https://opensource.microsoft.com/codeofconduct/faq/),如有任何疑问或意见可联系 opencode@microsoft.com。

熟悉贡献协议后,即可按照 NNI 开发人员教程,创建第一个 PR =)
熟悉贡献协议后,即可按照 NNI 开发人员教程,创建第一个 PR:

* 推荐新贡献者先找到标有 ['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) 或 ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) 标签的 Issue。这些都比较简单,可以从这些问题开始
* 推荐新贡献者先从简单的问题开始:['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) 或 ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22)。
* [NNI 开发环境安装教程](docs/zh_CN/Tutorial/SetupNniDeveloperEnvironment.md)
* [如何调试](docs/zh_CN/Tutorial/HowToDebug.md)
* 如果有使用上的问题,可先查看[常见问题解答](https://github.com/microsoft/nni/blob/master/docs/zh_CN/Tutorial/FAQ.md)。如果没能解决问题,可通过 [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) 联系 NNI 开发团队或在 GitHub 上 [报告问题](https://github.com/microsoft/nni/issues/new/choose)。
* [自定义 Tuner](docs/zh_CN/Tuner/CustomizeTuner.md)
* [实现定制的训练平台](docs/zh_CN/TrainingService/HowToImplementTrainingService.md)
* [在 NNI 上实现新的 NAS Trainer](https://github.com/microsoft/nni/blob/master/docs/zh_CN/NAS/NasInterface.md#implement-a-new-nas-trainer-on-nni)
Expand All @@ -349,7 +326,7 @@ You can use these commands to get more information about the experiment
* [使用 NNI 为 SPTAG 自动调参](docs/zh_CN/CommunitySharings/SptagAutoTune.md)
* [使用 NNI 为 scikit-learn 查找超参](https://towardsdatascience.com/find-thy-hyper-parameters-for-scikit-learn-pipelines-using-microsoft-nni-f1015b1224c1)
* **博客** - [AutoML 工具(Advisor,NNI 与 Google Vizier)的对比](http://gaocegege.com/Blog/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/katib-new#%E6%80%BB%E7%BB%93%E4%B8%8E%E5%88%86%E6%9E%90) 作者:[@gaocegege](https://github.com/gaocegege) - kubeflow/katib 的设计与实现的总结与分析章节
* **Blog (中文)** - [NNI 2019 新功能汇总](https://mp.weixin.qq.com/s/7_KRT-rRojQbNuJzkjFMuA) by @squirrelsc
* **博客** - [NNI 2019 新功能汇总](https://mp.weixin.qq.com/s/7_KRT-rRojQbNuJzkjFMuA) by @squirrelsc

## **反馈**

Expand Down
10 changes: 5 additions & 5 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ jobs:
yarn eslint
displayName: 'Run eslint'
- script: |
python3 -m pip install torch==0.4.1 --user
python3 -m pip install torchvision==0.2.1 --user
python3 -m pip install torch==1.2.0 --user
python3 -m pip install torchvision==0.4.0 --user
python3 -m pip install tensorflow==1.13.1 --user
python3 -m pip install keras==2.1.6 --user
python3 -m pip install gym onnx --user
Expand Down Expand Up @@ -91,8 +91,8 @@ jobs:
echo "##vso[task.setvariable variable=PATH]${HOME}/Library/Python/3.7/bin:${PATH}"
displayName: 'Install nni toolkit via source code'
- script: |
python3 -m pip install torch==0.4.1 --user
python3 -m pip install torchvision==0.2.1 --user
python3 -m pip install torch==1.2.0 --user
python3 -m pip install torchvision==0.4.0 --user
python3 -m pip install tensorflow==1.13.1 --user
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null
brew install swig@3
Expand Down Expand Up @@ -131,7 +131,7 @@ jobs:
- script: |
python -m pip install scikit-learn==0.20.0 --user
python -m pip install keras==2.1.6 --user
python -m pip install https://download.pytorch.org/whl/cu90/torch-0.4.1-cp36-cp36m-win_amd64.whl --user
python -m pip install torch===1.2.0 torchvision===0.4.1 -f https://download.pytorch.org/whl/torch_stable.html --user
python -m pip install torchvision --user
python -m pip install tensorflow==1.13.1 --user
displayName: 'Install dependencies'
Expand Down
Loading