July 2019 Release Endgame Plan

[suiguoxin]

This is the endgame schedule for July 2019 Iteration.
last updated on 8/4/2019

End game master: @suiguoxin
Aug 6: Code freeze for the endgame
Aug 12: Endgame done
Aug 12: Expected release date
release track @scarlett2018
refer to June endgame: #1161

08/06

• Code freeze at 4pm UTC+8
• Cut branch v1.0
• Build NNI test pypi package.
• Build NNI test docker image for v1.0 release bug bash.

08/12 Bug bash Round 2

• UT fail
• randint @ultmaster tester @suiguoxin
• quniform in metis tester
• smac support quniform only when q=1 @leckie-chn
• Failed trials show a huge negative duration on macOS Failed trials show a huge negative duration on macOS #1416 tester @QuanluZhang
• Set TPE classArgs as optional Set TPE classArgs as optional #1436 @SparkSnail tester @suiguoxin
• Hotfix for /tmp/nni permission problem Hotfix for /tmp/nni permission problem #1439 @liuzhe-lz tester @LeonardoWang
• fix bugs, sort, concurrency, multi fix bugs, sort, concurrency, multi #1451 tester @liuzhe-lz @LeonardoWang
• Add a parallel algorithm to improve the performance of TPE with large concurrency Add a parallel algorithm to improve the performance of TPE with large concurrency #1052 [need an option to turn on/off parallel] @xuehui1991 @Crysple tester @Chi song
• NNI doesn't support private docker registry NNI doesn't support private docker registry #755 @SparkSnail tester @QuanluZhang
• broken link tester @QuanluZhang

---

TODO in bug bash

• yarn.lock security alert @liuzhe-lz Eject from react-scripts-ts-antd and bump webui dependencies version #1412 tester @lvybriage
• Add NAS examples as NNI example Add NAS examples as NNI example #1158 @QuanluZhang
• Update filter description and fix typo
• IT fail @SparkSnail
• nnictl stop --all @SparkSnail
• It cannot work with nni-0.9 version but work with nni-0.7 version # issues1353 #pr1418
• randint @ultmaster @suiguoxin
• quniform in metis @suiguoxin
• Failed trials show a huge negative duration on macOS Failed trials show a huge negative duration on macOS #1416
• Set TPE classArgs as optional Set TPE classArgs as optional #1436 @SparkSnail
  new pr:
• [x]Hotfix for /tmp/nni permission problem Hotfix for /tmp/nni permission problem #1439 @liuzhe-lz
• smac support quniform only when q=1 @suiguoxin tester @leckie-chn

Docs, Blogs, Samples, Articles

• Update NNI Introduction Board Update NNI Introduction Board #1270 # 1313 @rabbit008
• Add a document similar to naming SupportTrainingService.md Add a document similar to naming SupportTrainingService.md #1370 @rabbit008 @SparkSnail

06/08-08/12 Bug bash Round 1

Plan Items

• Unify nas searchspace Unify nas searchspace #1393 @Crysple tester @ultmaster
• retrieve the values of the metrics in a programmatic way retrieve the values of the metrics in a programmatic way #868 @chicm-ms tester @suiguoxin
• Add a parallel algorithm to improve the performance of TPE with large concurrency Add a parallel algorithm to improve the performance of TPE with large concurrency #1052 [need an option to turn on/off parallel] @xuehui1991 @Crysple tester @Chi song
• support multiphase for hyperband PR support multi-phase in hyperband #1257 @QuanluZhang tester @chicm-ms

Tuners and Assessors

• Auto-Feature generator & selection Auto Feature Extraction #877 @baomengjiao @xuehui1991 tester @leckie-chn
  • design auto feature interface
  • random tuner or beam search tuner
  • pakdd example

Training Service

• NNI doesn't support private docker registry NNI doesn't support private docker registry #755 @SparkSnail tester @QuanluZhang
• get_experiment_id, get_trial_id python API @suiguoxin tester @liuzhe-lz

Engineering Improvements

• Python wrapper for rest api Python wrapper for rest api #1318(pr) @chicm-ms tester @suiguoxin
• sometimes CUDA_VISIBLE_DEVICES="-1" is not wanted sometimes CUDA_VISIBLE_DEVICES="-1" is not wanted #1365 @SparkSnail tester @QuanluZhang
• Remove outputDir and dataDir configuration in PAI mode Remove outputDir and dataDir in PAI mode #1342 @SparkSnail
• improvement: not Copy codeDir to logDir when create trial on Kubeflow mode Copy codeDir to logDir when create trial on Kubeflow mode  #1224 @SparkSnail tester @SparkSnail

Web Portal & User Experience

• Can nni show the best metric curve during search progress in WebUI? Can nni show the best metric curve during search progress in WebUI? #1218 @lvybriage (modify) tester @liuzhe-lz @LeonardoWang
• Some improvements in webui Some improvements in webui #1210 @lvybriage tester @liuzhe-lz @LeonardoWang
  • trial progress(intermediate results count) : as a customized column
• webui not support search parameters value webui not support search parameters value #1208 @lvybriage tester @liuzhe-lz @LeonardoWang
• WebUI上显示的metric动态调整下画图的区间 WebUI上显示的metric可以动态调整下画图的区间吗？ #1360 @lvybriage tester @liuzhe-lz @LeonardoWang
• Add a detailed documentation link to the nnictl command in the command prompt. Add a detailed documentation link to the nnictl command in the command prompt. #1260 @SparkSnail tester @suiguoxin
• webui nav webui nav  #1230 @lvybriage tester @liuzhe-lz @LeonardoWang
• UX improvement for showing Error log UX improvement for showing Error log #1173 @yangou1988 @Ivybriage tester @liuzhe-lz @LeonardoWang

Docs, Blogs, Samples, Articles

• Broken link check (build log) tester @leckie-chn @QuanluZhang @ultmaster tester @QuanluZhang
• multi phase doc improvement NAS mode doc improvement #1233 @chicm-ms #pull 1242
• Update the docs structure Update the docs structure #1231 @xuehui1991

Bug Fixes

• script for auto-complete @liuzhe-lz tester @SparkSnail
• Fix pipeline issue that it only check exit code of last command in a script. Fix pipeline #1417 @chicm-ms tester @SparkSnail
• quniform fors tuners quniform fors tuners #1377 @suiguoxin tester @ultmaster
• 'quniform' has different meaning beween GridSearch and other tuner. 'quniform' has different meaning beween GridSearch and other tuner. #1335 @suiguoxin tester @ultmaster
• "nnictl experiment list" give the status of a "RUNNING" experiment as "INITIALIZED" @SparkSnail tester @QuanluZhang
• SMAC cannot be installed if nni is installed in dev mode SMAC cannot be installed if nni is installed in dev mode #1376 @SparkSnail tester @suiguoxin
• 404 links in 0.9 release 404 links in 0.9 release #1236 @leckie-chn tester @rabbit008
• The filter button of the intermediate result cannot be clicked The filter button of the intermediate result cannot be clicked #1263 @lvybriage tester @liuzhe-lz @LeonardoWang
• API "/api/v1/nni/trial-jobs/xxx" doesn't show a trial's all parameters in multiphase experiment API "/api/v1/nni/trial-jobs/xxx" doesn't show a trial's all parameters in multiphase experiment #1258 @chicm-ms tester @lvybriage
• Succeeded trial doesn't have final result but webui show ×××(FINAL) Succeeded trial doesn't have final result but webui show ×××(FINAL) #1207 @chicm-ms tester @liuzhe-lz @LeonardoWang
• IT for nnictl stop @Crysple add IT for nnictl stop  #1298
• fix security warning @chicm-ms
• Hyper-parameter page broken Hyper-parameter page broken #1332 @lvybriage tester @liuzhe-lz @LeonardoWang
• Run flake8 tests to find Python syntax errors and undefined names Run flake8 tests to find Python syntax errors and undefined names #1217