Tag Archives: tensorflow

Error reporting of importing tensorflow package from pycharm

 [ Note] pycharm imports the tensorflow package and reports an error

If you also encounter this problem, it means you don’t understand where tensorflow is

After installing Anaconda 3.6, set interpreter in pycharm. This interpreter determines how to execute the code you write in pycharm environment

If your setting is python.exe under anaconda. You will find that import tensorflow is written in pycharm   When as TF, an error will be reported, indicating that there is no tensorflow module. This is because python.exe in Anaconda file is version 3.6, and the CONDA environment you created is under anaconda, and the tensorflow you installed is in CONDA environment, not Anaconda environment. Therefore, if you want to import tensorflow, you need to select python.exe in CONDA environment as the interpreter

When your interpreter selects python.exe under Anaconda in the red box shown in the figure below, you will be prompted that the tensorflow module is not found when importing tensorflow in pycharm environment

 

If you want to use tensorflow, the interpreter should choose the following path (Anaconda – > envs-> tenfowflow-> Python.exe under python. Exe)

The correct selection of the interpreter is shown in the figure. Click the setting button indicated by the arrow in the figure below, select add local, and select python.exe of the path shown in the figure above instead of python.exe under anaconda. After selecting python.exe under CONDA environment, the settings shown in the red box in the figure below will be displayed

 

Tensorflow encountered importerror: could not find ‘cudart64_ 100. DLL ‘error resolution

Explain the function of static keyword and final keyword in Java in detail>>>

When installing tensorflow, there was an error that DLL file could not be found when using import tensorflow. Referring to many solutions of blogs and stackflow, we found that only the version number did not match, but it did not specify what version was the right one. Therefore, we wrote this pit avoiding guide. Thank you again for your guidance and help

Author’s environment:

Python version 3.6

Tensorflow version 1.14

  ImportError: Could not find ‘cudart64_ 100.dll’

Brief answer:

Carefully analyze the types and causes of errors

Find out your tensorflow and CUDA version

Using the corresponding version to solve the problem, complete the adaptation of CUDA and TF, cudnn and CUDA, protobuf and TF

I. causes of error types

The problem is that we can’t find the module of CUDA system DLL file, so we need to download cuda10.0. First, check whether the file exists in CUDA’s path

Access CUDA through the path of C:// program files/NVIDIA GPU computing toolkit/CUDA, and find out whether cudart64 exists in its bin directory_ 100.dll module

If yes, check whether the environment variable is added; If not, it may be the matching problem between CUDA version and tensorflow version

2. Find out your tensorflow and CUDA version

Enter the command line environment, first through Python — version to determine their own Python version is 3.6

Then check the installed tensorflow version through PIP list. The version of the author is 1.14

Check CUDA version through nvcc — version. The previous CUDA version is v9.0.176

Find the corresponding version information through the official website of tesnsorflow

You can see that when tensorflow’s version & gt= 1.13, CUDA version needs to be 10.0, and cudnn version number needs to be greater than 7.4.1

Here I choose to unload CUDA version to apply tensorflow version

Directly enter the path of C:// program files/NVIDIA GPU computing toolkit/CUDA to delete the folder and environment variables

Found a high school students to build the server download link, here the download speed will be faster:

How much is circumcision in Wuxi http://www.bhnkyixue.com/

The corresponding cudnn corresponds to CUDA version

After installing CUDA version, open jupyter again and run import tensorflow. It is found that it is not successful, and the ‘cudnn64’ is not found_ Error in 7. DLL ‘:

  ImportError: Could not find ‘cudnn64_ 7.dll’

This prompt indicates that the DLL file of cudnn module is missing. According to the tensorflow document, cudnn should be > 7.4.1. Download cudnn. The directory structure of cudnn is as follows:

  

Put the files in cudnn directory in CUDA directory

4. Match the corresponding protobuf to the corresponding TF version

At this time, there should be no problem. The author continued to run import tensorflow, MMP, but it did not run smoothly. There was a “descriptor” error

  ImportError: cannot import name ‘descriptor’

Summary of TF installation problems on stackflow

Through searching on stackflow, it is found that the reason for this error is that the versions of protobuf and TF do not correspond, because there is a dependency relationship between TF and pro. Therefore, the author will first install pro, then install TF, and finally re install TF. TF will automatically install the dependency pro

There was a little episode in the middle. The author used virtualenv’s py virtual environment, so after installing the version, the error of ‘descriptor’ still appeared, so I tested it in the native py environment and found that TF can be imported. The reason is that the system can’t find the sitepackage in the PY virtual environment. If the directory in py bin of the virtual environment is set as the environment variable, it can be called normally

The final result is imported successfully

  

Tensorflow error due to uninitialized variable [How to Fix]

 

Initial code

import pandas as pd
import numpy as np
import tensorflow as tf train_input_data = pd.read_excel('new_data/4.12.2.xlsx',header=None,sheetname='train') train_input_data = np.array(train_input_data).ravel() train_input_data = list(train_input_data) train_input_data = tf.constant(train_input_data,dtype=tf.float32) train_output_data = train_input_data test_input_data = pd.read_excel('new_data/4.12.2.xlsx',header=None,sheetname='test') test_input_data = np.array(test_input_data).ravel() test_input_data = list(test_input_data) test_input_data = tf.constant(test_input_data,shape=[270,1],dtype=tf.float32) test_output_data = test_input_data print(type(train_input_data)) fc_mean, fc_var = tf.nn.moments(train_input_data,axes=[0]) scale = tf.Variable(tf.ones([1])) shift = tf.Variable(tf.zeros([1])) epslion = 0.001 train_input_data = tf.nn.batch_normalization(train_input_data,fc_mean,fc_var,shift,scale,epslion) sess = tf.Session() sess.run(train_input_data)

Error:

Traceback (most recent call last):
  File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
    return fn(*args)
  File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn status, run_metadata) File "/Users/zhangjia/anaconda/lib/python3.6/contextlib.py", line 89, in __exit__ next(self.gen) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status pywrap_tensorflow.TF_GetCode(status)) tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable_1 [[Node: Variable_1/read = Identity[T=DT_FLOAT, _class=["loc:@Variable_1"], _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_1)]] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/zhangjia/PycharmProjects/太阳能预测/test.py", line 30, in <module> a = sess.run(train_input_data) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run run_metadata_ptr) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1124, in _run feed_dict_tensor, options, run_metadata) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run options, run_metadata) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable_1 [[Node: Variable_1/read = Identity[T=DT_FLOAT, _class=["loc:@Variable_1"], _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_1)]] Caused by op 'Variable_1/read', defined at: File "/Users/zhangjia/PycharmProjects/太阳能预测/test.py", line 21, in <module> shift = tf.Variable(tf.zeros([1])) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 199, in __init__ expected_shape=expected_shape) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 330, in _init_from_args self._snapshot = array_ops.identity(self._variable, name="read") File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1400, in identity result = _op_def_lib.apply_op("Identity", input=input, name=name) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op op_def=op_def) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op original_op=self._default_original_op, op_def=op_def) File "/Users/zhangjia/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__ self._traceback = self._graph._extract_stack() # pylint: disable=protected-access FailedPreconditionError (see above for traceback): Attempting to use uninitialized value Variable_1 [[Node: Variable_1/read = Identity[T=DT_FLOAT, _class=["loc:@Variable_1"], _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_1)]]

Error reason:

in the initial code, because tf.global is not used_ variables_ Initializer () function to initialize variables, resulting in errors in code

Solution:

Add tf.global to the code_ variables_ After initializer () function is used to initialize the data, the problem of

can be solved

Correct code

import pandas as pd
import numpy as np
import tensorflow as tf train_input_data = pd.read_excel('new_data/4.12.2.xlsx',header=None,sheetname='train') train_input_data = np.array(train_input_data).ravel() train_input_data = list(train_input_data) train_input_data = tf.constant(train_input_data,dtype=tf.float32) train_output_data = train_input_data test_input_data = pd.read_excel('new_data/4.12.2.xlsx',header=None,sheetname='test') test_input_data = np.array(test_input_data).ravel() test_input_data = list(test_input_data) test_input_data = tf.constant(test_input_data,shape=[270,1],dtype=tf.float32) test_output_data = test_input_data fc_mean, fc_var = tf.nn.moments(train_input_data,axes=[0]) scale = tf.Variable(tf.ones([1])) shift = tf.Variable(tf.zeros([1])) epslion = 0.001 train_input_data = tf.nn.batch_normalization(train_input_data,fc_mean,fc_var,shift,scale,epslion) sess = tf.Session() init = tf.global_variables_initializer() sess.run(init)
data, vocabulary = prepare_test_data(config)
model = CaptionGenerator(config)
init = tf.global_variables_initializer()
sess.run(init)
model.load(sess, FLAGS.model_file)
tf.get_default_graph().finalize()
model.test(sess, data, vocabulary)

Error in calling GPU by keras or tensorflow: blas GEMM launch failed

Why can’t you stop buying 618?From the technical dimension to explore>>>

The GPU version of tensorflow encountered blas GEMM launch failed error during model training, or keras encountered the same error (keras generally takes tensorflow as the backend, if the GPU version of tensorflow is installed, GPU will be preferred when using keras)

InternalError (see above for traceback): Blas GEMM launch failed : a.shape=(300, 1), b.shape=(1, 10), m=300, n=10, k=1

This is when calling GPU, the video memory allocation encountered a problem. The way to compare security is to allocate memory space for tensorflow or keras before model training. Tensorflow creates a session with the following statement

gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333)  
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))

The parameters of keras are set when keras is introduced

import tensorflow as tf
from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.allocator_type = 'BFC' #A "Best-fit with coalescing" algorithm, simplified from a version of dlmalloc.
config.gpu_options.per_process_gpu_memory_fraction = 0.3
config.gpu_options.allow_growth = True
set_session(tf.Session(config=config))

If you use IPython notebook, GPU sync failed may appear after completing the above settings, and it should be OK to restart it

From: https://blog.csdn.net/Leo_ Xu06/article/details/82023330

I tried the useful of tensorflow, record it

Tensorflow reported an error when using session module: attributeerror: module ‘tensorflow’ has no attribute ‘session’, which has been solved

Geeks, please accept the hero post of 2021 Microsoft x Intel hacking contest>>>

After installing tensorflow 2.0, when using session, an error is reported: module ‘tensorflow’ has no attribute ‘session’:

Source code:

import tensorflow as tf
import os
os.environ["CUDA_VISIBLE_DEVICES"]="0"
a=tf.constant(2)
b=tf.constant(3)
with tf.Session() as sess:
    print("a:%i" % sess.run(a),"b:%i" % sess.run(b))
    print("Additionwithconstants:%i" % sess.run(a+b))
    print("Multiplicationwithconstant:%i" % sess.run(a*b))

Error message:

The error means that the tensorflow module does not have the session attribute. Later, we found that there is no session attribute in tensorflow 2.0. If you are installing tensorflow 2.0 and want to use the session attribute, you can change TF. Session() to:

tf.compat.v1.Session()

This method can solve this kind of problem, not only for session attribute

When running again, the program reported another error:

The reason is that version 2.0 is not compatible with version 1.0

tf.compat.v1.disable_eager_execution()

It’s going to work

The official website of tensorflow_ eager_ The execution () method is interpreted as follows:

This function can only be called before any Graphs, Ops, or Tensors have been created. 
It can be used at the beginning of the program for complex migration projects from TensorFlow 1.x to 2.x.

translation: this function can only be called before creating any graph, operation or tensor. It can be used at the beginning of a complex migration project from tensorflow 1. X to 2. X

Error in importing tensorflow in Jupiter Notebook: no module named tensorflow solution

1. Background

First of all, my environment:

os : windows10
anaconda版本:2.7

The official website provides two ways to install TensorFlow: PIP and anaconda. I use the anaconda method. After the installation is completed, you can import TensorFlow or run the program on the command line. Then I use the command Jupiter notebook to start Jupiter notebook in the created environment named TensorFlow. However, there is only python2 in the options of Jupiter notebook, and there is no newly installed environment, As shown in the following figure:

and when Import tensorflow as TF is executed, an error will occur: no module named TensorFlow

2. Solutions

Enter the anaconda command line, and then select TensorFlow environment:

> activate tensorflow

Execute the following command:

> conda install nb_conda

After the installation, start jupyter Notebook:

> jupyter notebook

At this time, you can see that the options for creating a file include our established environment TensorFlow

.

If you select TensorFlow environment and then Import tensorflow as TF , no error will occur

 

Solution for oserror: [errno 1] operation not permitted encountered when installing tensorflow

Knowledge map advanced must read: read how large-scale map data efficient storage and retrieval>>>

Installing collected packages: numpy, scipy, six, pyyaml, Keras, opencv-python, h5py, html5lib, bleach, futures, wheel, werkzeug, markdown, protobuf, tensorflow-tensorboard, pbr, funcsigs, mock, backports.weakref, tensorflow
  Found existing installation: numpy 1.14.1
    Uninstalling numpy-1.14.1:
      Successfully uninstalled numpy-1.14.1
  Found existing installation: scipy 0.13.0b1
    DEPRECATION: Uninstalling a distutils installed project (scipy) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling scipy-0.13.0b1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 778, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 754, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-k6sIvY-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy-0.13.0b1-py2.7.egg-info'

  

First of all, I think of the problem of directory permissions, so I don’t know how many times’ Chmod 755 ‘has been implemented, and it doesn’t work

Is that because there is something wrong with the ownership of the directory?It’s a new way to modify the “chown usename” and it’s still useless

Finally, we had to Google and found this article: https://www.jianshu.com/p/7a18c78b5982 Yes, he solved my problem. He just needs a little upgrade to the old package. Here is a record of the executed commands as follows:

$ pip install --upgrade pip

$ sudo pip install numpy --upgrade --ignore-installed
$ sudo pip install scipy --upgrade --ignore-installed
$ sudo pip install scikit-learn --upgrade --ignore-installed

  

Tensorflow Warning: cona The environment is inconsistent, please check the package plan carefully

No solution, but it does not seem to affect the use of

(py36) C:\Users\LEEG>conda install numpy
Collecting package metadata: done
Solving environment: |
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::anaconda==4.4.0=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::astropy==1.3.2=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::blaze==0.10.1=py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::bottleneck==1.2.1=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::dask==0.14.3=py36_1
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::distributed==1.16.3=py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::matplotlib==2.0.2=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::numba==0.33.0=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::numexpr==2.6.2=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::odo==0.5.0=py36_1
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::pandas==0.20.1=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::patsy==0.4.1=py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::pytables==3.2.2=np112py36_4
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::pywavelets==0.5.2=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::scikit-image==0.13.0=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::scikit-learn==0.18.1=np112py36_1
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::scipy==0.19.0=np112py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64::seaborn==0.7.1=py36_0
– https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/win-64::statsmodels==0.8.0=np112py36_0

Tensorflow: tensorflow.python.framework.errors_impl.NotFoundError (dataset_ops.so not found)

Operating Environment

Windows 10

Python3.6

Tensorflow 1.9.0

Error Codes:

tf.add_to_collection('losses', tf.contrib.layers.l2_regularizer(regularizer)(w))

Error

tensorflow.python.framework.errors_impl.NotFoundError: D:\Users\Administrator\Anaconda3\lib\site-packages\tensorflow\contrib\data\python\ops…._dataset_ops.so not found

Solution

Go to the tensorflow/contrib/data directory and delete **_dataset_ops.so**

Solution to GPU memory leak problem of tensorflow operation efficiency

Problem Description:

Tensorflow runs slower and slower during training, and it gets better after restarting.

I used Tensorflow-GPU version 1.2, and I ran on the GPU. When I started training, the time of each batch was very low. Then as the training progressed, the time of each batch was getting longer and longer, but when I After restarting, everything is normal again?

Problem finding:

  The reason I found at the beginning was the batch_size and batch_num problems. It was solved by the python yield data generator to ensure that the data processed by the memory each time was determined to be the batch_size size, but it was found that the operating efficiency was still not high, so I checked some information on Google and found the following solutions Method.

problem solved:

It is caused by the definition of tf’s op in the runtime session. In this way, each iteration will add a new node in the graph, resulting in a memory leak, the program is getting slower and slower, and finally it is forced to exit. As for whether the program adds nodes during runtime, you can define graph.finalize() in the session to lock the graph. If an error is reported when running, it proves that the program is getting slower and slower by dynamically adding nodes.

The code before modification is as follows:

def one_hot(labels):
    labels_num = [strnum_convert(i) for i in labels]
    batch_size = tf.size(labels_num)
    labels = tf.expand_dims(labels_num, 1 )
    indices = tf.expand_dims(tf.range(0, batch_size, 1), 1 )
    concated = tf.concat([indices, labels],1 )
    onehot_labels = tf.sparse_to_dense(concated, tf.stack([batch_size, 8]), 1 , 0)
     # all_hot_labels = tf.reshape(onehot_labels,(1,612)) 
    return onehot_labels

The modified code is as follows:

def one_hot(labels):
    one_hot_label = np.array([int(i == int(labels)) for i in range(8 )])    
    ... ...
     return one_hot_label

You can see that the culprit is the one_hot operation of this tf version, which is modified to the numpy version to perfectly solve the problem of operating efficiency.

Thinking:

Method Two:

The cause of the above problem is GPU memory leak, we can also use a curve to save the country; every 1000 batches, when the speed is significantly slower, reset the graph, and then rebuild the model, and then save before load The parameter tf.reset_default_graph()self.build_model();

Method three:

When we used tensorflow to make a data set, we found that when I run the eval() function, the program will run slower and slower. The value generated by eval() is not deleted, and then it will occupy more memory. The solution is Just use the del command, usually written as.

data = Var.eval()  
 # save data to file 
del data