Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with Fillers in matcaffe in OSX #178

Closed
sguada opened this issue Feb 27, 2014 · 7 comments
Closed

Problem with Fillers in matcaffe in OSX #178

sguada opened this issue Feb 27, 2014 · 7 comments

Comments

@sguada
Copy link
Contributor

sguada commented Feb 27, 2014

This is a weird problem.
When using imagenet_deploy.txt containing weight_filler and/or bias_filler in a layer then the net.init() process don't continue.

When not using weight_filler and/or bias_filler in the layers then net.init() works well but then when restoring the trained model there is dimension mismatch in fc6 layer.

@shelhamer
Copy link
Member

Sergio, you already raised this issue in #175. Please close one of these.

@shelhamer shelhamer reopened this Feb 27, 2014
@shelhamer
Copy link
Member

Also, I'm afraid you're not even on dev since imagenet_deploy.prototxt doesn't exist anymore, having been replaced by models/imagenet.prototxt. Or is this bug on master?

@sguada
Copy link
Contributor Author

sguada commented Feb 27, 2014

@shelhamer sorry for the confusion with imagenet_deploy.prototxt and imagenet.prototxt. I know there is no imagenet_deploy.prototxt anymore, but imagenet.prototxt doesn't currently work (since it include the fillers) with matcaffe, so I created a temporal imagenet_deploy.prototxt which doesn't include fillers, but then I get a problem when restoring the net due to the dimensions.

@sguada
Copy link
Contributor Author

sguada commented Feb 27, 2014

@shelhamer I'm tracking down the error, it seems that it is related to OSX, since in Ubuntu it doesn't happen.

If one removes the fillers from imagenet.prototxt then matcaffe_demo works in OSX, otherwise it doesn't. Although it takes a lot of time to set_mode to CPU (I cannot test GPU due to memory constraints), while it should be fast since class Caffe is a singleton (isn't it? @Yangqing). These problems don't happen in Ubuntu.

Once the mode is set it is fast to process the images.

@sguada
Copy link
Contributor Author

sguada commented Feb 27, 2014

@Yangqing the part of the code that make it slow is the initialization of CUBLAS in common.cpp

  if (cublasCreate(&cublas_handle_) != CUBLAS_STATUS_SUCCESS) {
    LOG(ERROR) << "Cannot create Cublas handle. Cublas won't be available.";
  }

Does anyone knows why this happen? This makes matlab to use 300MB extra of memory. When in reality it is never used in CPU mode.
This is something to keep in mind for CPU-ONLY version #3

@shelhamer
Copy link
Member

@sguada is this still a problem or ancient history?

@shelhamer
Copy link
Member

Closing; pretty sure this is solved by matcaffe improvements and the resolution of the CUDA set device call order issue in #507 #521.

lukeyeager pushed a commit to lukeyeager/caffe that referenced this issue Jun 27, 2016
Multi-GPU support for the GPUMemory::Workspace
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants