Tag Archives: GPU

Error in calling GPU by keras or tensorflow: blas GEMM launch failed

Why can’t you stop buying 618?From the technical dimension to explore>>>

The GPU version of tensorflow encountered blas GEMM launch failed error during model training, or keras encountered the same error (keras generally takes tensorflow as the backend, if the GPU version of tensorflow is installed, GPU will be preferred when using keras)

InternalError (see above for traceback): Blas GEMM launch failed : a.shape=(300, 1), b.shape=(1, 10), m=300, n=10, k=1

This is when calling GPU, the video memory allocation encountered a problem. The way to compare security is to allocate memory space for tensorflow or keras before model training. Tensorflow creates a session with the following statement

gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333)  
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))

The parameters of keras are set when keras is introduced

import tensorflow as tf
from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.allocator_type = 'BFC' #A "Best-fit with coalescing" algorithm, simplified from a version of dlmalloc.
config.gpu_options.per_process_gpu_memory_fraction = 0.3
config.gpu_options.allow_growth = True
set_session(tf.Session(config=config))

If you use IPython notebook, GPU sync failed may appear after completing the above settings, and it should be OK to restart it

From: https://blog.csdn.net/Leo_ Xu06/article/details/82023330

I tried the useful of tensorflow, record it

“Failed to get convolution algorithm. This is probably because cuDNN failed to initialize”

You’ve recently been using the version of inf.0. The following error occurred while running the program.

Failed to get convolution algorithm. This is probably because cuDNN failed to initialize

At first, it was suspected that CUDA and cudnn were configured incorrectly (version matching is required). After trial and error, there is still this error

from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSession

config = ConfigProto()
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)

It means to allocate GPUs on demand
the main reason is that my image is relatively large and consumes more GPU resources. But my graphics card (rtx2060) has only 6GB memory, so this error will appear. This error prompt is very misleading, which makes people obsess about the versions of CUDA and cudnn. Therefore, I’ll stick it here to prevent future generations from repeating the same mistakes.

reference resources:

https://github.com/tensorflow/tensorflow/issues/24828