Expose all netstate options (for all-in-one nets) #3863

lukeyeager · 2016-03-21T21:52:14Z

There are 3 configurable dimensions to a network's NetState: phase (TRAIN or TEST), level and stage. Currently, only phase is exposed through common interfaces. This PR exposes level and stage to the user, further enabling all-in-one nets (#1245)

New CLI:

build/tools/caffe time -model network.prototxt -level 1 -stage deploy

New pycaffe interface:

net = caffe.Net('network.prototxt', weights='weights.caffemodel',
  phase=caffe.TEST, level=1, stages=['deploy'])

EDIT: PR has been updated not to break old code

~~This is a breaking change (see #1790 (comment) for an explanation). Users will need to transform pycaffe code like:~~

# old
net = caffe.Net('network.prototxt', 'weights.caffemodel', caffe.TEST)

~~to:~~

# new
net = caffe.Net('network.prototxt', weights='weights.caffemodel', phase=caffe.TEST)
# or
net = caffe.Net('network.prototxt', caffe.TEST, weights='weights.caffemodel')

With some effort, this could probably be done without breaking the interface (but I'd need some help with the Boost.Python code). However, I am actually in favor of breaking the interface in order to make it simpler and more "Pythonic".

seanbell · 2016-03-25T18:13:48Z

While I agree that the new interface is more Pythonic, I personally think that the old interface can be maintained with a little bit of massaging. All that needs to be edited is the Net.__init__ function, which can be done in the same style as the rest of the _Net_X functions.

Modify the end of pycaffe.py by adding:

# Original Boost.Python Net.__init__ function
_Net_init_boost = Net.__init__


def _Net_init(self, *args, **kwargs):
    """ Construct a caffe.Net """
    # Transform the arguments to handle the legacy API
    if len(args) == 2 and not kwargs:
        # Handle case: Net(param_file, phase)
        _Net_init_boost(self, args[0], weights_file=None, phase=args[1])
    elif len(args) == 3 and not kwargs:
        # Handle case: Net(param_file, weights_file, phase)
        _Net_init_boost(self, args[0], weights_file=args[1], phase=args[2])
    else:
        # Handle new API:
        # Net('net.prototxt', weights_file='weights.caffemodel', phase=caffe.TEST, level=1, stages=['deploy'])
        _Net_init_boost(self, *args, **kwargs)


# Attach methods to Net.
Net.__init__ = _Net_init
...

The above code makes make pytest pass, but I haven't tested it thoroughly.

One other note: It seems that if you wanted to use a kwarg for every argument, then in some place it's named param_file and in others it's named network_file. Maybe this should be harmonized to use the same name?

ajtulloch · 2016-04-17T09:20:23Z

include/caffe/net.hpp

@@ -25,6 +25,7 @@ class Net {
 public:
  explicit Net(const NetParameter& param, const Net* root_net = NULL);
  explicit Net(const string& param_file, Phase phase,
+      const int level = 0, const vector<string>* stages = NULL,


Could you also insert the new arguments (level, stages) after root_net? Otherwise, this breaks calling code that instantiates Net with root_net non-null:

Net(param_file, phase, root_net);

I'm not sure how common this is, but it's pretty trivial to avoid it by just inserting the new optional arguments at the end of the declaration?

I agree, that seems like the safer choice. I wonder why no tests failed? Seems like a test hole.

Does anyone use the root_net parameter? I'd like to keep the phase, level and stages parameters next to each other since they're naturally grouped. If anything, I'd prefer to move root_net before phase, but that would definitely break existing code.

As per Evan's comments below, I reverted back to the original behavior of moving root_net to the end.

lukeyeager · 2016-04-18T22:34:56Z

One other note: It seems that if you wanted to use a kwarg for every argument, then in some place it's named param_file and in others it's named network_file. Maybe this should be harmonized to use the same name?

I agree, the naming change is annoying. network_file and weights_file seem more meaningful to me than param_file and pretrained_param_file. They do a better job of conveying the fact that the first file is a textual representation of the network, while the second file is a binary file containing numerical weights.

I thought it might be appropriate to rename them here since this is the transition from internal code to external interface. Other common interfaces use words like solver.net=... and caffe -weights=....

EDIT - Oh, I see what you were pointing out. I was being internally inconsistent with my own code. Whoops!

lukeyeager · 2016-04-18T23:57:10Z

While I agree that the new interface is more Pythonic, I personally think that the old interface can be maintained with a little bit of massaging. All that needs to be edited is the Net.__init__ function, which can be done in the same style as the rest of the _Net_X functions.

@seanbell I think I've found a nice compromise between our solutions. I've left the old Net_Init_Load() function in place, and I'm using Boost.Python's pattern matching to distinguish between Net(string, string, int) and Net(string, int, int).

Now all of these constructors work as expected:

### Legacy style
net = caffe.Net('network.prototxt', caffe.TRAIN)
net = caffe.Net('network.prototxt', 'weights.caffemodel', caffe.TEST)
  # pattern matches to Net(network_file, weights_file, phase)
  # prints deprecation warning

### New style
net = caffe.Net('network.prototxt', phase=caffe.TRAIN)
net = caffe.Net('network.prototxt', weights_file='weights.caffemodel', phase=caffe.TEST)
net = caffe.Net('network.prototxt', phase=caffe.TEST, weights_file='weights.caffemodel')
net = caffe.Net('network.prototxt', caffe.TRAIN, 0)
  # pattern matches to Net(network_file, phase, level)
net = caffe.Net('network.prototxt', caffe.TEST, 0, [], 'weights.caffemodel')

Made changes and rebased to address merge conflict from #3982.

lukeyeager · 2016-04-26T01:12:51Z

Is there anything in particular holding back this PR?

Does the re-arranging of the root_net parameter break any of the @BVLC team's crazy [cool] workflows?

Is my Python constructor compromise not satisfactory?

lukeyeager · 2016-05-13T22:43:15Z

Updated Net constructor to not switch order of arguments - good idea @ajtulloch
Added -phase flag to tools/caffe so that you can time the TEST phase
Rebased to master

ajtulloch · 2016-05-14T01:35:40Z

LGTM. Cool stuff @lukeyeager!

shelhamer · 2016-05-14T04:24:03Z

Will review after NIPS deadline 05/20. Thanks for this @lukeyeager and thanks for review @ajtulloch!

shelhamer · 2016-05-25T21:57:16Z

python/caffe/_caffe.cpp

+          bp::default_call_policies(), (bp::arg("network_file"), "phase",
+            bp::arg("level")=0, bp::arg("stages")=bp::object(),
+            bp::arg("weights_file")=bp::object())))
+    // XXX Legacy constructor


No need for XXX -- the comment is clear as it is.

shelhamer · 2016-05-25T22:12:05Z

Re: root_net in #3863 (comment) and #3863 (comment), this is a pretty obscure arg that should never really be used outside of the framework itself. Furthermore I hope it can be eventually refactored out through further work on the parallel implementation. With that in mind, I think it is reasonable to keep it as the last, optional arg as to not complicate the normal instantiation of Nets. Thoughts?

shelhamer · 2016-05-25T23:10:06Z

Although I originally voted for net states and these varied ways of expressing them, I'm now of two minds. On one hand phase, level, and stage states make all-in-one definitions possible and potentially make variations easier to express, but on the other these are just more special cases in the definition of nets. That said, they do seem to address a need in practice and since they are in the library they should at least be properly exposed to its interfaces. Thoughts @jeffdonahue @longjon?

While I now use and suggest net spec, net states have their purposes so thanks for extending them throughout the interfaces @lukeyeager.

For merge:

consider arg name weights_file vs. weights
extend caffe train to accept level, stage or decide not to and warn/die if they are given
migrate net state tests to C++ side of testing, or at least add a basic test there to not rely on Python
switch order of args with root_net?

Let me know if there are other concerns.

lukeyeager · 2016-05-25T23:26:34Z

I've addressed (1) and (2).

(3) Gulp. Time to dive into gtest ...

(4) Don't care

lukeyeager · 2016-06-01T17:26:41Z

make sure to update solver.cpp accordingly:

None of those invocations need updating - they use a different constructor which expects a NetParameter object instead of a filename.

lukeyeager · 2016-06-01T17:45:44Z

I'd rather switch the order of root_net

Done

It would be best to have a C++ test that checks that the layers for do in fact exist

Done

shelhamer · 2016-06-01T17:49:02Z

#3863 (comment) Right, sorry about that.

With #3863 (comment) this looks good to me. @longjon will double-check for merge.

This internal functionality will be exposed through the various interfaces in subsequent commits Also adds C++ tests for all-in-one nets

Adds command-line flags for phase, level and stage train -- override level and stages for test_state from solver test -- set level and stages time -- set phase, level and stages

Uses Boost.Python's pattern matching to differentiate between constructors Also adds Python tests for all-in-one nets

lukeyeager · 2016-06-03T22:36:38Z

Rebased after conflict from #4227

lukeyeager · 2016-06-15T21:04:28Z

this looks good to me. @longjon will double-check for merge.

ping @longjon

shelhamer · 2016-07-11T05:04:36Z

Merging after offline discussion. Thanks @lukeyeager!

p.s. @jeffdonahue noted that this not strictly necessary since the NetState of the NetParameter can be set before instantiating a Net. However, this PR contributes a one-step interface to instantiating any state and enables exposing the different states to languages lacking protobuf bindings (such as matlab, as far as I know).

lukeyeager · 2016-07-11T16:44:58Z

this not strictly necessary since the NetState of the NetParameter can be set before instantiating a Net.

Sure, but it's nice to be able to change it without editing .prototxt.

Merging after offline discussion.

Thanks!

…-options Expose all netstate options (for all-in-one nets)

…ukeyeager/bvlc/expose-all-netstate-options" This reverts commit 3e94c0e, reversing changes made to f28f5ae.

This was referenced Mar 21, 2016

Expose net stages to pycaffe #3736

Closed

[Discussion] How to specify all-in-one networks #3864

Open

shelhamer mentioned this pull request Mar 22, 2016

Give phase to Net #1790

Merged

shelhamer added interface JD JL ES labels Mar 22, 2016

lukeyeager force-pushed the bvlc/expose-all-netstate-options branch from 670ac9a to 7b2638f Compare March 22, 2016 16:31

ajtulloch reviewed Apr 17, 2016
View reviewed changes

lukeyeager force-pushed the bvlc/expose-all-netstate-options branch 2 times, most recently from a8d1c46 to 7a162bf Compare April 18, 2016 23:48

lukeyeager force-pushed the bvlc/expose-all-netstate-options branch from 7a162bf to 0daee4a Compare April 19, 2016 00:09

shelhamer added the focus label Apr 26, 2016

lukeyeager force-pushed the bvlc/expose-all-netstate-options branch from 0daee4a to 8397212 Compare May 13, 2016 22:08

lukeyeager mentioned this pull request May 19, 2016

Update summarize.py to select different netstate options #4174

Closed

shelhamer reviewed May 25, 2016
View reviewed changes

lukeyeager force-pushed the bvlc/expose-all-netstate-options branch from 8397212 to dab822c Compare May 25, 2016 22:05

lukeyeager added 3 commits June 3, 2016 15:28

Add level and stages to Net constructor

d167e61

This internal functionality will be exposed through the various interfaces in subsequent commits Also adds C++ tests for all-in-one nets

Add phase, level and stages to tools/caffe

66e84d7

Adds command-line flags for phase, level and stage train -- override level and stages for test_state from solver test -- set level and stages time -- set phase, level and stages

Add level and stages to pycaffe

19adc7a

Uses Boost.Python's pattern matching to differentiate between constructors Also adds Python tests for all-in-one nets

lukeyeager force-pushed the bvlc/expose-all-netstate-options branch from 56c316e to 19adc7a Compare June 3, 2016 22:36

shelhamer added ready for review and removed ES JD JL labels Jun 9, 2016

lukeyeager mentioned this pull request Jun 27, 2016

draw net doesn't understand net phases #4365

Closed

lukeyeager mentioned this pull request Jul 5, 2016

Add phase support for draw net #4408

Merged

shelhamer merged commit 3e94c0e into BVLC:master Jul 11, 2016

lukeyeager deleted the bvlc/expose-all-netstate-options branch July 11, 2016 16:45

This was referenced Jul 12, 2016

Get stats & original network def from "Download Model" button NVIDIA/DIGITS#891

Merged

Replace input fields with a proper layer #1245

Closed

Convert examples to all-in-one networks #4458

Closed

lukeyeager mentioned this pull request Aug 11, 2016

Request - integrate code for all-in-one nets NVIDIA/caffe#118

Closed

fxbit pushed a commit to Yodigram/caffe that referenced this pull request Sep 1, 2016

Merge pull request BVLC#3863 from lukeyeager/bvlc/expose-all-netstate…

b82abb9

…-options Expose all netstate options (for all-in-one nets)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose all netstate options (for all-in-one nets) #3863

Expose all netstate options (for all-in-one nets) #3863

lukeyeager commented Mar 21, 2016 •

edited

Loading

seanbell commented Mar 25, 2016

ajtulloch Apr 17, 2016

lukeyeager Apr 18, 2016

lukeyeager Apr 18, 2016

lukeyeager Jun 1, 2016

lukeyeager commented Apr 18, 2016 •

edited

Loading

lukeyeager commented Apr 18, 2016

lukeyeager commented Apr 26, 2016

lukeyeager commented May 13, 2016

ajtulloch commented May 14, 2016

shelhamer commented May 14, 2016

shelhamer May 25, 2016 •

edited

Loading

shelhamer commented May 25, 2016

shelhamer commented May 25, 2016 •

edited

Loading

lukeyeager commented May 25, 2016

lukeyeager commented Jun 1, 2016

lukeyeager commented Jun 1, 2016

shelhamer commented Jun 1, 2016

lukeyeager commented Jun 3, 2016

lukeyeager commented Jun 15, 2016

shelhamer commented Jul 11, 2016

lukeyeager commented Jul 11, 2016

Expose all netstate options (for all-in-one nets) #3863

Expose all netstate options (for all-in-one nets) #3863

Conversation

lukeyeager commented Mar 21, 2016 • edited Loading

seanbell commented Mar 25, 2016

ajtulloch Apr 17, 2016

Choose a reason for hiding this comment

lukeyeager Apr 18, 2016

Choose a reason for hiding this comment

lukeyeager Apr 18, 2016

Choose a reason for hiding this comment

lukeyeager Jun 1, 2016

Choose a reason for hiding this comment

lukeyeager commented Apr 18, 2016 • edited Loading

lukeyeager commented Apr 18, 2016

lukeyeager commented Apr 26, 2016

lukeyeager commented May 13, 2016

ajtulloch commented May 14, 2016

shelhamer commented May 14, 2016

shelhamer May 25, 2016 • edited Loading

Choose a reason for hiding this comment

shelhamer commented May 25, 2016

shelhamer commented May 25, 2016 • edited Loading

lukeyeager commented May 25, 2016

lukeyeager commented Jun 1, 2016

lukeyeager commented Jun 1, 2016

shelhamer commented Jun 1, 2016

lukeyeager commented Jun 3, 2016

lukeyeager commented Jun 15, 2016

shelhamer commented Jul 11, 2016

lukeyeager commented Jul 11, 2016

lukeyeager commented Mar 21, 2016 •

edited

Loading

lukeyeager commented Apr 18, 2016 •

edited

Loading

shelhamer May 25, 2016 •

edited

Loading

shelhamer commented May 25, 2016 •

edited

Loading