Error report tensorflow/stream_ executor/cuda/cuda_ dnn.cc:338] Could not create cudnn handle: CUDNN_ STATUS_ INTERNAL_ ERROR

Error code

2019-05-05 09:34:42.298865: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-05-05 09:34:42.469839: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-05-05 09:34:43.086143: E tensorflow/stream_executor/cuda/cuda_dnn.cc:338] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2019-05-05 09:34:43.100978: E tensorflow/stream_executor/cuda/cuda_dnn.cc:338] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR

Solutions

First of all, it’s a big hole. I’ve found many ways on the Internet, basically changing the GPU runtime config. I find that running some programs doesn’t work; There are also requests to downgrade CUDA and cudnn versions. I tried to downgrade CUDA 10.1 all the way to 9.0, and cudnn has been changed all the time. The error still exists, and it still doesn’t work

In order to solve this problem, I try to install tensorflow GPU with docker. It is found that the program can run successfully, but it is not very convenient. I find that Anaconda uses CONDA to install tensorflow GPU in a virtual environment. Tensorflow GPU will automatically install CUDA and cudnn in this environment, and will not affect CUDA and cudnn already installed outside the ring, but the default version is relatively low, so, The best way is to specify the version to install. You only need to specify the version of tensorflow, and the versions of CUDA and cudnn will correspond

Install in virtual environment

conda install tensorflow-gpu==1.12

This simple command has been torturing for several days. CUDA cudnn tensorflow has been tried in various versions. On GitHub, many people say that they are waiting for the new version of tensorflow to be repaired. Indeed, this method can only solve the problem temporarily, and can’t migrate to a higher version, such as tensorflow 2.0, so for the time being, it’s better to use 1.12