Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

NAS for TensorFlow #2115

Merged
merged 42 commits into from
Apr 26, 2020
Merged

NAS for TensorFlow #2115

merged 42 commits into from
Apr 26, 2020

Conversation

liuzhe-lz
Copy link
Contributor

@liuzhe-lz liuzhe-lz commented Mar 3, 2020

I will re-generate the commit history to make this patch clear. Expected commit order is shown as follow.
The items in itatic font are not yet ready.
Some features may be moved to future PRs.

  • mutable interface
  • mutator interface
  • ENAS
  • ENAS-specific static graph
  • DARTS
  • DARTS-specific static graph
  • docstring
  • Proxyless NAS (under evaluation)
  • General static graph interface
  • Port static graph interface to model compression
  • Other NAS algorithms


_counter = 0

def _global_mutable_counting():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this counting makes it hard to do key matching of two independent running. For example, one is training and the other is applying.

@QuanluZhang
Copy link
Contributor

better to add docstring

@ultmaster
Copy link
Contributor

ultmaster commented Mar 23, 2020

The code looks almost identical to that of PyTorch. Can we have some diffchecker that can diff tensorflow directory against pytorch directory? To me, another draft PR overwriting the pytorch folder would be great.

@liuzhe-lz liuzhe-lz marked this pull request as ready for review March 30, 2020 02:45
@liuzhe-lz liuzhe-lz changed the base branch from master to v1.5 March 30, 2020 23:58
def undedup_mutables(self):
return self._structured_mutables.traverse(deduplicate=False)

def forward(self, *inputs):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think forward is not needed for tensorflow.

self.reduction = reduction
self.return_mask = return_mask

def forward(self, *inputs):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call?

self.reduction = reduction
self.return_mask = return_mask

def forward(self, optional_inputs):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

test_arc_per_epoch = 1


class EnasTrainer:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an example ready for testing this trainer from end to end?

@ultmaster
Copy link
Contributor

ultmaster commented Mar 31, 2020

If no further improvement is planned for this PR, I'm strongly against merging it into master. Merging into a dev branch might be more acceptable.

The main reason is, code currently presented in this PR seems to be never tested from end to end, which makes me doubt whether current design can actually work on tensorflow.

@liuzhe-lz liuzhe-lz changed the base branch from v1.5 to dev-nas-tf April 1, 2020 00:50
@scarlett2018 scarlett2018 added this to the 2020 April-May Release milestone Apr 13, 2020
@liuzhe-lz liuzhe-lz marked this pull request as draft April 22, 2020 02:28
Lijiaoa and others added 2 commits April 23, 2020 17:34
* update

* update

* update

* update

* update log message in bug-report template

Co-authored-by: Lijiao <15910218274@163.com>
@liuzhe-lz liuzhe-lz marked this pull request as ready for review April 24, 2020 08:40
@liuzhe-lz liuzhe-lz merged commit 146c745 into microsoft:dev-nas-tf Apr 26, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.