CUDA driver 331.38 incompatibility

[shelhamer]

There is some kind of CUDA incompatibility with NVIDIA driver 331.38. To fix, downgrade to 331.20.

Original detective work by @rbgirshick :

---

We isolated it to an problem with the backwards pass of conv4. The problem manifested itself as GPU utilization dropping to ~3%, nothing happening for a while, and then the caffe process segfaulting.

---

As far as we know, this is a bug with the driver (Caffe works properly on all other versions tried).

[kloudkl]

Just for reference, the driver version in CUDA 5.5 is 319.37. Maybe upgrading should be delayed until CUDA 6.0 is released.

[kloudkl]

CUDA 6 Release Candidate (RC) is now available.

[shelhamer]

This driver is neither current nor the default with CUDA releases, so this warning is no longer needed.