Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Dev weight sharing update doc #577

Merged
Merged
Changes from all commits
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
fb8bc25
add pycharm project files to .gitignore list
leckie-chn Oct 9, 2018
0bf454c
update pylintrc to conform vscode settings
leckie-chn Oct 11, 2018
7f0a418
Merge remote-tracking branch 'upstream/master'
leckie-chn Oct 11, 2018
9fce9bb
Merge remote-tracking branch 'upstream/master'
leckie-chn Oct 17, 2018
73b9b58
Merge remote-tracking branch 'upstream/master'
Oct 18, 2018
d83aa70
Merge remote-tracking branch 'upstream/master'
leckie-chn Oct 19, 2018
16be159
Merge remote-tracking branch 'upstream/master'
leckie-chn Oct 25, 2018
69466b8
fix RemoteMachineMode for wrong trainingServicePlatform
leckie-chn Oct 25, 2018
f1f8339
Merge branch 'master' of https://github.com/leckie-chn/nni
leckie-chn Oct 25, 2018
7a1be57
simple weight sharing
leckie-chn Oct 25, 2018
99b1402
update gitignore file
leckie-chn Oct 25, 2018
253068d
change tuner codedir to relative path
leckie-chn Oct 25, 2018
2faac01
add python cache files to gitignore list
leckie-chn Oct 26, 2018
e412cb5
move extract scalar reward logic from dispatcher to tuner
leckie-chn Oct 26, 2018
e159afb
update tuner code corresponding to last commit
leckie-chn Oct 26, 2018
d664f4f
update doc for receive_trial_result api change
leckie-chn Oct 26, 2018
b5d5f2e
add numpy to package whitelist of pylint
leckie-chn Oct 26, 2018
3b280cd
distinguish param value from return reward for tuner.extract_scalar_r…
leckie-chn Oct 26, 2018
a384da0
update pylintrc
leckie-chn Oct 29, 2018
cd50908
Merge remote-tracking branch 'upstream/master' into dev-mac-support
Oct 29, 2018
5b1320a
add comments to dispatcher.handle_report_metric_data
leckie-chn Oct 29, 2018
3aee412
update install for mac support
Oct 29, 2018
60db733
fix root mode bug on Makefile
triflame92 Oct 29, 2018
094436d
Quick fix bug: nnictl port value error (#245)
SparkSnail Oct 18, 2018
8fca02e
Dev exp stop more (#221)
QuanluZhang Oct 18, 2018
cbc808b
update Makefile (#246)
Crysple Oct 18, 2018
ce17fa3
quick fix for ci (#248)
Crysple Oct 18, 2018
e337541
add update trialNum and fix bugs (#261)
Crysple Oct 23, 2018
07d51dd
Add builtin tuner to CI (#247)
Crysple Oct 23, 2018
0fbe564
Doc refactor (#258)
scarlett2018 Oct 23, 2018
e35f96d
Refactor nnictl to support listing stopped experiments. (#256)
SparkSnail Oct 23, 2018
71dc1ca
Show experiment parameters more beautifully (#262)
lvybriage Oct 24, 2018
5c65cef
fix error on example of RemoteMachineMode (#269)
leckie-chn Oct 25, 2018
07fe4ef
Update docker file to use latest nni release (#263)
chicm-ms Oct 25, 2018
dc688b8
fix bug about execDuration and endTime (#270)
QuanluZhang Oct 26, 2018
f8b131c
Refactor dockerfile (#264)
SparkSnail Oct 26, 2018
ec0c1d5
Support nnictl tensorboard (#268)
SparkSnail Oct 26, 2018
95d8666
Sdk update (#272)
chicm-ms Oct 26, 2018
a3b60cc
add experiment log path to experiment profile (#276)
chicm-ms Oct 30, 2018
a550686
Merge branch 'dev-mac-support' into dev-mac-support
leckie-chn Oct 30, 2018
d4c383a
refactor extract reward from dict by tuner
leckie-chn Oct 31, 2018
4ba1916
Merge remote-tracking branch 'upstream/master'
leckie-chn Nov 1, 2018
641f5d7
Merge remote-tracking branch 'upstream/master'
leckie-chn Nov 1, 2018
78e4209
Merge remote-tracking branch 'upstream/master'
leckie-chn Nov 8, 2018
909d9d9
Merge remote-tracking branch 'upstream/master'
leckie-chn Nov 12, 2018
221e951
Merge remote-tracking branch 'upstream/master'
leckie-chn Nov 13, 2018
cca83a2
Merge branch 'master' into dev-mac-support
leckie-chn Nov 13, 2018
5d001f7
Merge pull request #1 from leckie-chn/dev-mac-support
leckie-chn Nov 13, 2018
dd12229
Merge pull request #1 from Microsoft/master
chicm-ms Nov 13, 2018
8f696ac
update Makefile for mac support, wait for aka.ms support
triflame92 Nov 13, 2018
df36c1f
Merge remote-tracking branch 'upstream/master'
leckie-chn Nov 14, 2018
bedc6fd
Merge branch 'master' into dev-weight-sharing
leckie-chn Nov 14, 2018
3583b52
refix Makefile for colorful echo
leckie-chn Nov 14, 2018
8aeea2e
Merge branch 'master' into dev-weight-sharing
leckie-chn Nov 14, 2018
b9cdde5
unversion config.yml with machine information
leckie-chn Nov 14, 2018
ae979c9
sync graph.py between tuners & trial of ga_squad
leckie-chn Nov 16, 2018
001ecd5
sync graph.py between tuners & trial of ga_squad
leckie-chn Nov 16, 2018
a67a6b8
Merge pull request #2 from Microsoft/master
chicm-ms Nov 16, 2018
75fd2f1
Merge pull request #3 from Microsoft/master
chicm-ms Nov 19, 2018
10e998f
Merge pull request #4 from Microsoft/master
chicm-ms Nov 27, 2018
a0f361c
Merge pull request #5 from Microsoft/master
chicm-ms Nov 27, 2018
76d7142
Merge pull request #6 from Microsoft/master
chicm-ms Nov 28, 2018
bc10bf7
Merge pull request #7 from Microsoft/master
chicm-ms Nov 29, 2018
d76deb3
Merge pull request #8 from Microsoft/master
chicm-ms Dec 4, 2018
af1137c
copy weight shared ga_squad under weight_sharing folder
leckie-chn Dec 10, 2018
7086cb5
mv ga_squad code back to master
leckie-chn Dec 10, 2018
c20abca
Merge remote-tracking branch 'upstream/dev-weight-sharing_1' into dev…
leckie-chn Dec 10, 2018
a0fd7ed
simple tuner & trial ready
leckie-chn Dec 10, 2018
0ffc4b4
Fix nnictl multiThread option
chicm-ms Dec 11, 2018
b3a1131
Merge remote-tracking branch 'chicm/fix_multithread' into dev-weight-…
leckie-chn Dec 11, 2018
4788ad5
weight sharing with async dispatcher simple example ready
leckie-chn Dec 11, 2018
efac915
update for ga_squad
leckie-chn Dec 17, 2018
56105e0
fix bug
leckie-chn Dec 21, 2018
93b8afe
modify multihead attention name
leckie-chn Dec 24, 2018
c9c64e5
add min_layer_num to Graph
leckie-chn Dec 24, 2018
adb9b40
fix bug
leckie-chn Dec 24, 2018
9c15d1a
update share id calc
leckie-chn Dec 25, 2018
041099b
fix bug
leckie-chn Dec 25, 2018
e3ee26f
add save logging
leckie-chn Dec 25, 2018
a9b7e58
fix ga_squad tuner bug
leckie-chn Dec 26, 2018
b5675fa
sync bug fix for ga_squad tuner
leckie-chn Dec 26, 2018
db1d96f
fix same hash_id bug
leckie-chn Dec 27, 2018
20586e3
add lock to simple tuner in weight sharing
leckie-chn Jan 2, 2019
5228657
Add readme to simple weight sharing
leckie-chn Jan 2, 2019
3a0176e
update
leckie-chn Jan 2, 2019
a9f5d62
update
leckie-chn Jan 2, 2019
c6098ed
add paper link
leckie-chn Jan 2, 2019
2469f48
update
leckie-chn Jan 2, 2019
61bc21c
reformat with autopep8
leckie-chn Jan 4, 2019
85d3076
add documentation for weight sharing
leckie-chn Jan 4, 2019
9fc984e
test for weight sharing
leckie-chn Jan 4, 2019
ddbcead
delete irrelevant files
leckie-chn Jan 4, 2019
500852e
move details of weight sharing in to code comments
leckie-chn Jan 7, 2019
2c11b73
add example section
leckie-chn Jan 7, 2019
a759c24
merge with upstream
leckie-chn Jan 7, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/AdvancedNAS.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Tutorial for Advanced Neural Architecture Search
Currently many of the NAS algorithms leverage the technique of **weight sharing** among trials to accelerate its training process. For example, [ENAS][1] delivers 1000x effiency with '_parameter sharing between child models_', compared with the previous [NASNet][2] algorithm. Other NAS algorithms such as [DARTS][3], [Network Morphism][4], and [Evolution][5] is also leveraging, or has the potential to leverage weight sharing.

This is a tutorial on how to enable weight sharing in NNI.
This is a tutorial on how to enable weight sharing in NNI.

## Weight Sharing among trials
Currently we recommend sharing weights through NFS (Network File System), which supports sharing files across machines, and is light-weighted, (relatively) efficient. We also welcome contributions from the community on more efficient techniques.
Expand Down Expand Up @@ -63,6 +63,8 @@ The feature of weight sharing enables trials from different machines, in which m
self.events[parameter_id].set()
```

## Examples
For details, please refer to this [simple weight sharing example](../test/async_sharing_test). We also provided a [practice example](../examples/trials/weight_sharing/ga_squad) for reading comprehension, based on previous [ga_squad](../examples/trials/ga_squad) example.

[1]: https://arxiv.org/abs/1802.03268
[2]: https://arxiv.org/abs/1707.07012
Expand Down