Converting to and from cuda-convnet

[jamt9000]

Unfinished, but creating PR to start discussion.

I plan on creating a tool to convert between cuda-convnet (pickled) files and caffe protobuffers.

At the moment it can (try to) convert the cuda-convnet "schema" to prototxt format (ie., ignoring the blobs). Next I will copy the blobs too and work on going the other way. It also doesn't do anything sensible about the input and loss layers yet.

If you have the supplied decaf net (trained with cuda-convnet) you can try:

print cudaconv_to_prototxt('./imagenet.decafnet.epoch90')

[Yangqing]

Thanks @jamt9000 !

If it helps, the old decaf repo has some codes that tries to read and convert cuda-convnet formats. Of course decaf uses yet another network definition, but it may be helpful:

https://github.com/UCB-ICSI-Vision-Group/decaf-release/tree/master/decaf/util/translator

[shelhamer]

Note #219. If you convert to the current caffe.proto the tool in #219 will update it to the new schema we're adopting, but you may want to wait and convert directly to the new one.

[jamt9000]

Yeah, I'll switch to the new schema when that's all settled.

Also, I spent a day debugging before realising...the decaf net is upside down :| But now the conversion seems to work. Also, another nice surprise is that caffe imagenet wrapper net is BGR instead of RGB :)

[jamt9000]

Here's an example to show it working

Convert the net:

from convert_net import *

netpt = cudaconv_to_prototxt('/home/james/Code/decaf-release/imagenet/imagenet.decafnet.epoch90')
fh=open('/tmp/decafnet.caffe.prototxt', 'w')
fh.write(netpt)
fh.close()

netpb = cudaconv_to_proto('/home/james/Code/decaf-release/imagenet/imagenet.decafnet.epoch90')
fh=open('/tmp/decafnet.caffe', 'wb')
fh.write(netpb.SerializeToString())
fh.close()

Compare predictions on lena with DeCAF:

import caffe
from decaf.scripts.imagenet import DecafNet
from pylab import imread

# prepare image
imname = '/home/james/Desktop/lena.jpg' # 256x256
im = imread(imname)[16:-16,16:-16,:]    # 224x224
im = im - 128. # subtract "mean"
flippedim = np.require(im[::-1][None,...], np.float32, 'C')
flippedim2 = np.rollaxis(flippedim,3,1).copy() # channels first for caffe

# Predict with DeCAF
dnet = DecafNet('/home/james/Code/decaf-release/imagenet/imagenet.decafnet.epoch90', '/home/james/Code/decaf-release/imagenet/imagenet.decafnet.meta')
decaf_prediction = dnet.classify_direct(flippedim).mean(0)

# Predict with CAFFE
caffenet = caffe.CaffeNet('/tmp/decafnet.caffe.prototxt', '/tmp/decafnet.caffe')
caffenet.set_phase_test()
caffenet.set_mode_cpu()
output_blobs = [np.empty((1, 1000, 1, 1), dtype=np.float32)]
caffenet.Forward([flippedim2], output_blobs)
caffe_prediction = output_blobs[0].mean(0).flatten()

# Use top_k function from DeCAF
decaf_labels, decaf_names = dnet.top_k_prediction(decaf_prediction, 5)
caffe_labels, caffe_names = dnet.top_k_prediction(caffe_prediction, 5)

print 'DeCAF', decaf_names
print 'probs', decaf_prediction[decaf_labels]
print 'CAFFE', caffe_names
print 'probs', caffe_prediction[caffe_labels]


# DeCAF ['sombrero', 'cowboy hat', 'hand blower', 'bonnet', 'shower cap']
# probs [ 0.33793405  0.26088405  0.06967235  0.05376853  0.02327657]
# CAFFE ['sombrero', 'cowboy hat', 'hand blower', 'bonnet', 'shower cap']
# probs [ 0.33793321  0.26088339  0.06967238  0.05376886  0.02327651]

[Yangqing]

Thanks @jamt9000 for working this out, and I apologize for the decaf upside-down thing - yeah, it is a legacy issue and we were too lazy to address it...

[jamt9000]

I'll wait for the refactoring before working on writing to cuda-convnet format. But I'm wondering what to do about dropout, since it is left as an "exercise for the reader" in the official cuda-convnet and there are various different implementations around (eg dnouri's). Would BVLC be able to share the patch they use to allow for better comparison?

[shelhamer]

@jamt9000 the proto refactoring should be merged soon–thanks for your patience.

As for a dropout patch for cuda-convnet, there is no such patch from Caffe since it was developed as a separate project. @jeffdonahue is your JeffNet fork of cuda-convnet public?

Re: BGR, we have OpenCV's default to thank for that. Who would ever choose that?

[bhack]

This is the oldest PR in the queue. It is 7 months old. Why some PR like this seems abandoned?

• The proposer is still available?
• We are waiting for others contributors to come for finish this work?
• We are waiting for response from core team? This PR is no more interesting?

[Yangqing]

Please understand that we are a small research team doing this, and will
prioritize stuff based on actual needs - at the current moment, my
impression is that supporting cuda-convnet is not a top priority. Caffe has
all the training capabilities so there is little reason for us to invest
time on this.

If anyone would like to finish writing a translator I am happy to include
it in an unsupported/ folder. But keep in mind that our further development
may well break it.

Yangqing

On Wed, Nov 26, 2014 at 4:34 PM, bhack notifications@github.com wrote:

This is the oldest PR in the queue. It is 7 months old. Why some PR like
this seems abandoned?

• The proposer is still available?
• We are waiting for others contributors to come for finish this work?
• We are waiting for response from core team? This PR is no more
  interesting?

Reply to this email directly or view it on GitHub
#212 (comment).

[bhack]

@Yangqing I know. My question was on this older because I want to understand if the process of handing the PR queue could be improved and to incentive "third party" contributors in the period that they have effectively time to invest on.

[bhack]

Probably we could cluster PR using a common tag semantics so its always clear what is the "stage/status" of an open PR.

[ducha-aiki]

@Yangqing, may be caffe-core-dev group consider the possibility of adding one active community-member as moderator with rights to merge? It happens in almost all communities after library or whatever becomes popular.

[bhack]

I don't know what is the best solution every community has found its own. The actual approach surely doesn't scale.

[jamt9000]

I am very busy at the moment (just started a PhD) but should have time closer to Christmas and will surely be contributing to Caffe more in the future.

[bhack]

@jamt9000 never mind the life change. This is the motivation cause I suggest to maintain the time slot a little bit short on PR for not vast contributions and stay with unfinished code.

[shelhamer]

Closing as stale -- however model converters are welcome.

[plutolove]

Can you send ‘imagenet.decafnet.epoch90’ and ‘imagenet.decafnet.meta’ to me？
The download link is not available.

[dupf]

Can you send ‘imagenet.decafnet.epoch90’ and ‘imagenet.decafnet.meta’ to me？
The download link is not available.

[qiongxiao]

@jamt9000 do you have these two files of " imagenet.decafnet.epoch90" and "imagenet.decaf.meta"? they could not be downloaded from Yangqing Jia's homepage: http://daggerfs.com/index.html#publications.